
\documentclass[12pt]{article}
\usepackage[T2A,OT1]{fontenc}
\usepackage[default]{cantarell}
\usepackage[a4paper, top=20mm, bottom=20mm, left=20mm, right=20mm]{geometry}
\usepackage[utf8]{inputenc}
\usepackage[russian, english]{babel}
\usepackage{tabu}
\usepackage{hyperref}
\usepackage{parskip}
\usepackage{graphicx}
\usepackage{tabularx}
\usepackage[normalem]{ulem}
\usepackage{float}
\floatstyle{boxed}
\restylefloat{figure}
\usepackage{setspace}
\onehalfspacing
\author{Artyom Bologov \href{mailto:logical-pathnames@aartaka.me}{(email)}}
\date{\today}
\title{Lisp Logical Pathnames}
\makeatletter
\def\endenv{\expandafter\end\expandafter{\@currenvir}}
\makeatother
\begin{document}
\maketitle

\includegraphics[width=\textwidth,height=\textheight,keepaspectratio]{./assets/logical-pathnames.png}

Logical pathnames are a powerful feature of Common Lisp.
Abstracting system-specific filenames and locations behind a special hostname.
Making UNIX and Windows filenames easily representable with a common format.
Making build-specific directories discoverable and reusable, regardless of packaging.
Allowing weird shortcut names saving typing.
Logical pathnames (LP (not Linkin Park) from now) are cool.
Except...

\begin{itemize}\item LPs are under-specified and implementation-specific
\item LPs are syntactically restricted
\item There are no explanations of how to work with LPs on the Internet
\end{itemize}

So I (as any Lisper) decided to experiment with these obscure things.
And figure out how they actually work.
Including the gory details of how implementations do LPs and how to use them “portably”.

This post was prompted by
\href{https://n16f.net/blog/working-with-common-lisp-pathnames}{Nicolas Martyanoff’s post on pathnames}
and the mention of logical pathnames there.

\section*{Logical Pathname Syntax} \label{syntax}

Logical pathnames are host-prefixed semicolon-separated directory+file paths:

\begin{figure}[h!]
\begin{figure}[h!]\begin{verbatim}
;; Implementation-reserved SYS: host
#p"sys:encodings;tools"
;; CCL-specific CCL and HOME
#p"ccl:"
#p"home:ccl-init"
;; Imaginary logical pathnames, might be defined later
#p"junk:test.lisp"
#p"l:"
#p"\href{https://codeberg.org/aartaka/cl-minifloats/src/branch/main/cl-minifloats.asd}{git:cl-minifloats;cl-minifloats.asd}"
#p"\href{run:lisp-design-patterns}{site:lisp-design-patterns}"
</pre>
<figcaption> Examples of logical pathnames
</figure>

The only mandatory part is the host.
The rest should be directories (separated by semicolons) and files (regular \verb|name.type| conventions.)
One significant restrictions is that both directories and file names/extensions should only consist of letters, digits, and hyphens.
Nothing else.
Not even ASCII.
Mere hyphenated alphanumerics.
(Actually, SBCL and CCL allow arbitrary chars recognizing them as file name parts, while ECL and ABCL error out.)
(Actually, letters are supposed to be uppercase, and all lowercase ones are converted to uppercase automatically.
Quite some historic baggage...)

Which is a significant restriction.
But, given that we—people defining logical pathnames—are a programming minority
and have a luxury of control over our file naming...
We’re fine.

\section*{Match Patterns} \label{patterns}

But Logical Pathname path syntax is not the only type of syntax one encounters with these.
There are also match patterns that define what kind of LPs are valid for a host:

\begin{figure}[h!]\begin{verbatim}
"" ;; Works everywhere except ECL
"*.*"
"**;*.*"
"L:**;*.*.*"
"MAIL;**;*.MAIL"
"CODE;DOCUMENTATION.*.*"
\end{verbatim}\caption{Examples of match patterns}\end{figure}

Notice the abundance of asterisks there.
These stars are \verb|:wild| (\verb|*|) and \verb|:wild-inferiors| (\verb|**|) components to be replaced later.
Replaced with the content following the host in actual instantiated pathnames.
So logical pathnames are actually a small pattern-matching
\href{run:cl-is-lots}{DSL (one among the lots others)}
transforming paths.
Now to actual use:

\section*{Translation APIs} \label{translation}

There are only three entry points defining how logical pathnames are processed:

\begin{description}\item[logical-pathname-translations]
to define (set) translations,
\item[translate-logical-pathname]
to resolve these,
\item[and load-logical-pathname-translations]
to do host-loading magic you’ll likely never need.
\end{description}

And that’s it.
The rest is handled by Common Lisp internals, resolving logical pathnames when actually using them.
(Maybe calling \verb|translate-logical-pathname| too?)
So the most interesting thing about LP API is \verb|logical-pathname-translations|.
Its setter (yes, it’s a \verb|setf|-able accessor (?)) defines new translation from the \verb|setf|-ed pattern-translation list:

\begin{figure}[h!]\begin{verbatim}
(setf (logical-pathname-translations "l")
      `(("L:**;*.*" "/home/aartaka/junkyard/test.lisp")))
\end{verbatim}\caption{Simple logical pathname definition}\end{figure}

\paragraph{OS portability} \begin{quote}
  Note that I’m using a UNIX-like notation of forward slash separated dirs/files.
  Which makes my logical pathnames less portable than they might be.
  If I aimed for portability across Linux, Mac, and Windows, I’d rather do something like \verb|(merge-pathnames (make-pathname :directory '(:relative "junkyard") :name "test" :type "lisp") (user-homedir-pathname))| so that directory separators are OS-specific.
    But this will do for the sake of example and because I mainly use Linux.
  \href{https://codeberg.org/aartaka/lisp-config/commit/887e7b3}{Wait, I actually did that when moving to Mac.}
\end{quote}

Now, all this syntax looks off, so I’m going to define a macro:

\begin{figure}[h!]\begin{verbatim}
(defmacro define-logical-pathname (host (pattern expansion) &rest patterns)
  `(setf (logical-pathname-translations ,host)
         (list (list ,pattern ,expansion)
               ,@(loop for (pattern expansion) in patterns
                       collect `(list ,pattern ,expansion)))))
;; Better now
(define-logical-pathname "l" ("L:**;*.*.*" "/home/aartaka/junkyard/test.lisp"))
\end{verbatim}\caption{Logical pathname definition macro}\end{figure}

So what happens is that:

\begin{enumerate}\item We take a host to add translation to;
\item We provide a match pattern (or several of them);
\item Plus the actual path to resolve to;
\item And \verb|setf| it to be the (set of) patterns for the given host.
\end{enumerate}

\subsection*{Simple Translation} \label{simple}

This LP I defined is supposed to be a shortcut to an individual file.
So it will be optimal to use it as \verb|l:|.
While also minimizing the pattern.
So how far can we get the match pattern to retain this constant property?

\begin{figure}[h!]\begin{verbatim}
;; All OK
(define-logical-pathname "l" ("L:**;*.*.*" "/home/aartaka/junkyard/test.lisp")) ;; Version element
(define-logical-pathname "l" ("L:**;*.*" "/home/aartaka/junkyard/test.lisp")) ;; No version
(define-logical-pathname "l" ("**;*.*" "/home/aartaka/junkyard/test.lisp")) ;; Omitted obvious host
;; OK, but only for empty or file-only paths (l:file.type)
(define-logical-pathname "l" ("*.*" "/home/aartaka/junkyard/test.lisp"))
;; OK, but only for empty or extension-less file paths (l:file)
(define-logical-pathname "l" ("*" "/home/aartaka/junkyard/test.lisp"))
;; OK, but only for empty paths (l:) and not everywhere
(define-logical-pathname "l" ("" "/home/aartaka/junkyard/test.lisp"))
\end{verbatim}\caption{Shortening substitutions}\end{figure}

Most of these are overkill for a single-file LP.
But the range and the possibilities are clear: you can pattern-match paths of varying complexity with these.


\subsection*{Substituting Translation} \label{substituting}

The fun stuff starts when you substitute path parts into the pattern:

\begin{figure}[h!]\begin{verbatim}
(define-logical-pathname "a" ("**;*.*" "/home/aartaka/**/*.*"))
;; Same, a standard shortcut for file omission?
(define-logical-pathname "a" ("**;*.*" "/home/aartaka/**/"))

(translate-logical-pathname #p"a:web;logical-pathnames.htm")
;; => #P"/home/aartaka/web/logical-pathnames.htm"
\end{verbatim}\caption{Simple $HOME-resolving logical pathname}\end{figure}

Notice the symmetry: we take \verb|:wild|/\verb|*| and \verb|:wild-inferiors|/\verb|**| from the pattern.
And plug them into the translation.
That’s the superpower: we can substitute nested paths.
Well, as long as there’s not too many wild parts there.

The overall heuristic seems to be:

\begin{itemize}\item \verb|**| is replaced with the arbitrary matching path in the provided logical pathname.
\item \verb|*| in directories seem to be replaced in the order of appearance.

\begin{figure}[h!]\begin{figure}[h!]\begin{verbatim}
(define-logical-pathname "X" ("X:a;*;b;*;*.*" "/hello/*/hi/*/what/*.*"))

(translate-logical-pathname #p"x:a;bonjour;b;barev;greetings.me")
;; => #P"/hello/bonjour/hi/barev/what/greetings.me"
</pre><figcaption> Multiple directory asterisks
</figure>

\item Asterisks in file name, type, and version (I know, no one uses versions, which is a shame)
  are replaced with these from the provided logical pathname.
\item The rest is matched and replaced verbatim.
\end{itemize}

Oh, right!
We can also provide multiple patterns to match paths against,
to be matched in the order of appearance:

\begin{figure}[h!]\begin{verbatim}
(define-logical-pathname "a"
    ("W;**;*.*" "/home/aartaka/web/**/*.*")
    ("G;**;*.*" "/home/aartaka/git/**/*.*")
    ("**;*.*" "/home/aartaka/**/"))

(translate-logical-pathname #p"a:w;logical-pathnames.htm")
;; => #P"/home/aartaka/web/logical-pathnames.htm"
\end{verbatim}\caption{Multi-pattern logical pathname (notice the dispatch prefix, it’s a frequent pattern!)}\end{figure}

What if I tried doing something smarter, like repeating the same wildcard twice in replacement?

\begin{figure}[h!]\begin{verbatim}
(define-logical-pathname "a"
    ("G;*.asd" "/home/aartaka/git/*/*.asd"))

(translate-logical-pathname #p"a:g;cl-minifloats.asd")
;; SBCL:
;;  :WILD-INFERIORS is not paired in from and to patterns:
;;  (:ABSOLUTE "G") (:ABSOLUTE "home" "aartaka" "git" :WILD)
;; ECL:
;;  #P"A:G;CL-MINIFLOATS.ASD" is not a specialization of path #P"A:G;*.ASD"
;; CCL:
;;  Error: No translation for #P"a:g;cl-minifloats.asd"
;; ABCL:
;;  Unsupported case in TRANSLATE-DIRECTORY-COMPONENTS.
\end{verbatim}\caption{Uh oh, no overly smart things}\end{figure}

So we’re finally hitting the limitations of logical pathnames.
They are powerful, but not in the Turing-complete way like
\href{run:cl-is-lots}{other DSLs in CL}.
Use caution.

\par\noindent\rule{\textwidth}{0.4pt}

Logical pathnames are a nice idea with a lot of potential.
But they are under-specified and implementations are inconsistent at times.
To summarize the behavior:

\begin{itemize}\item Logical pathnames are a DSL for file path resolution.
\item These are logical host plus semicolon-separated hyphenated alphanumeric directories and files.
\begin{itemize}\item Gotcha: some implementations allow other chars, but that’s not portable.</ul>
\item These get resolved according to the rules provided to \verb|logical-pathname-translations|.
\begin{itemize}\item Gotcha: rules are resolved in the order they are provided in.</ul>
\item Rules have a left hand side (pattern to match) and right hand side (resulting path.)
\begin{itemize}\item Gotcha: Left and right hand side should (not must—that’s implementation-specific) have matching wild components.</ul>
\item These wild elements match the parts of the actual logical pathname and get inserted into the resulting path.
\begin{itemize}\item One by one and strictly once per wildcard.</ul>
\item Translation happens in \verb|translate-logical-pathname|, often implicit in using pathname APIs.
\item Logical pathnames are cool, albeit quite restricted.
\end{itemize}

So yeah, you can use logical pathnames for their many benefits, as long as you stick to portable patterns.
I’m going to use some of these in my setup:

\begin{figure}[h!]\begin{verbatim}
#-ccl
(define-logical-pathname "home"
    ("HOME:**;*.*.*" "/home/aartaka/**/*.*"))

(define-logical-pathname "cfg"
    ("CFG:**;*.*.*" "/home/aartaka/.config/common-lisp/**/*.*"))

(define-logical-pathname "git"
    ("GIT:**;*.*.*" "/home/aartaka/git/**/*.*"))
\end{verbatim}\caption{My potential logical pathnames}\end{figure}


\par\noindent\rule{\textwidth}{0.4pt}
\href{https://creativecommons.org/licenses/by/4.0}{CC-BY 4.0} 2022-2026 by Artyom Bologov (aartaka,)
\href{https://codeberg.org/aartaka/pages/commit/a91befa}{with one commit remixing Claude-generated code}.
Any and all opinions listed here are my own and not representative of my employers; future, past and present.
\end{document}
