These thoughts came about during a conversation with a friend of mine. They should be taken as reading advice for first-time readers of Jorge Luis Borges.
If you haven’t read Borges, you should. There’s a collection called Labyrinths, which contains a nice selection of both his fiction and non-fiction. It’s perhaps the best place to start. The advantage of having a selection of Borges’s essays in the same volume as his stories is that the essays are, simultaneously, a lively counterpoint to the stories and a clear illumination of the ideas, obsessions, and themes at their heart. From there you could go to his Selected Non-Fictions, edited by Eliot Weinberger (as Borgesian an editor as you could wish for and as eruditely imaginative an essayist as Borges himself*) and then on to the single-volume Collected Fictions, translated by Andrew Hurley (this is controversial terrain; see here). Both these are available as handsome, deckle-edged, Deluxe editions from Penguin.
Most of Borges’s stories read like essays and non-fiction accounts, while his non-fiction possesses the creativity and imagination we associate with the best of fiction. And they’re all centred around logical or philosophical puzzles of one kind or another. In my opinion though—maybe because of the currently available translations, maybe not—Borges the essayist is more magical than Borges the fictioneer. And even if you choose not to read his fiction, you certainly should, nay must, read his fantastic ‘non-fictions’.