The term “web directory” is older than most of the web that a reader in 2026 actually uses, and it is regularly confused with two neighbouring things — the search engine and the business directory — from which it deserves to be separated with some care. This article offers a definition rather than a list, because the definition is the part most accounts hurry past on their way to recommendations. It sets out what a web directory is, what problem it was built to address, where the form came from, and — since the question is a fair one — what purpose and what value the web directory retains in 2026, a year in which the way people find anything online is once again unsettled.
A word on sources. The claims made here about why a directory of this kind exists, and about how finding information online imposes real costs, are drawn from peer-reviewed research in economics and information science, cited by author and year and listed at the end. The observations about the state of search in 2026 rest on industry reporting rather than peer-reviewed evidence, and where I rely on them I say so.
A working definition of the web directory
A web directory is a catalogue of websites, organized by subject into categories, and arranged so that a person can find a site by moving through those categories rather than by typing a query. Each entry in a classical web directory is not a web page in the abstract but a whole site, placed by a human editor under a heading that describes what the site is about, often with a short written summary. The definition has three parts worth separating. It is a catalogue, which is to say a deliberately assembled and bounded collection rather than the whole of the web. It is organized by subject, which is to say its structure is a classification — a hierarchy of topics — and not an index of words. And it is browsable, which is to say the intended way to use it is to descend through categories from the general to the specific, in the manner of someone walking through a library rather than calling out a question.
It is useful to mark, at once, the distinction between a web directory and a business directory, since the two terms are close enough to be muddled. A business directory lists organizations; a web directory lists websites. The two overlap, because many websites belong to businesses and many web directories contain a commercial branch, but the organizing intention differs. The business directory answers the question of which organization can perform a service; the web directory answers the broader and older question of which sites, on any subject at all, are worth knowing about. This article concerns the second.
It is worth being concrete about what a single directory entry actually consists of, since the entry is the directory’s basic unit. A classical entry holds, at minimum, the name of the site, a link to it, and the category under which it has been placed; most directories add a short description, written by an editor or supplied by the site and then checked, that tells a reader what the site offers before they visit it. That description is not a trivial addition. It is the directory doing, in miniature, the work this article describes — converting the raw fact that a site exists into a piece of organized, summarized, human-assessed information. The point also marks the directory off from a simple list of links, with which it is sometimes confused. A list of links records that its author found certain pages worth keeping; a directory additionally classifies what it records and describes it for a reader who is not its author. The presence of structure and description, and not merely the presence of links, is what makes a directory a directory.
The kinds of web directory
The single definition given above covers several forms that differ enough in practice to be worth separating, because a reader who has encountered only one of them may carry assumptions that do not hold for the rest. The broadest distinction is between the general directory and the niche directory. A general directory sets out to cover every subject — the arts, the sciences, commerce, recreation, and all the others — within one hierarchy; this was the form taken by the great catalogues of the early web, and it is the form whose characteristic difficulties the later sections of this article examine. A niche, or vertical, directory abandons that ambition on purpose and covers one subject only — a single industry, a single profession, a single field of interest — in greater depth than any general directory could afford it. The distinction is not cosmetic, because it changes what the directory can honestly promise: a bounded subject can be catalogued thoroughly and kept current, whereas the whole web cannot.
A second distinction is geographic. A regional or local directory restricts itself not by subject but by place, listing the sites, and often the organizations behind them, attached to a particular city, country, or area. Such a directory exchanges universal coverage for local completeness, and for a user whose need is itself local that exchange is usually worth making. A third distinction concerns how a site comes to be listed, and whether money changes hands in the process. Some directories accept submissions without charge and admit a site purely on an editor’s assessment of it; others operate a paid-submission model, in which the owner of a site pays a fee for consideration, or for the listing itself. The paid model is not improper in principle, but it introduces a tension — examined later in this article — between the directory’s role as an impartial filter and its interest in the revenue that listings produce. There is, finally, the loosely used term “link directory”, applied, often with a note of disapproval, to directories whose entries exist less to guide a human reader than to manufacture links pointing at the listed sites; the rise and fall of that form has its own section below. None of these kinds is rigidly separate from the others, and a real directory commonly combines several of them, but holding them apart lets a reader see that “web directory” names a family of related things rather than one uniform object.
The problem a web directory was built to solve
The scale of the early web
The web directory is best understood as a response to a specific difficulty, and the difficulty is one of scale. By the late 1990s the web had already grown past the point at which any person could hold a useful mental map of it, and the structure of the thing made the problem worse rather than better. The large-scale study of the web’s link structure by Broder and colleagues (2000) found a graph that was vast, only loosely connected, and without any centre from which the rest could be reached — a shape that makes undirected wandering an unreliable way to find anything. A reader who knew that a good site on a given subject existed somewhere had, in practice, no dependable means of arriving at it. The web had outgrown the methods of finding things that had served when it was small.
The cost of finding, and the value of a filter
Economists had described the underlying problem long before the web existed. Stigler (1961), in the work that founded the economics of information, established that information is costly to acquire: a person seeking something must spend real effort — time, attention, repeated enquiry — in the search, and any institution that lowers that effort has genuine value. Bakos (1997), carrying the argument into electronic markets, showed that reducing the buyer’s cost of search changes how such a market behaves. A web directory is, in the plainest terms, an institution for lowering the cost of finding a website, since it concentrates in one organized place what a person would otherwise have to assemble by trial and error.
There is a second function, distinct from the first. A directory does not merely gather sites; it gathers sites that a human editor judged worth listing. This matters because the web, then as now, contained a great deal that was low in quality or actively misleading, and a searcher could not always tell the difference in advance. The problem is the one Akerlof (1970) analysed in his account of markets with asymmetric information: when a person cannot assess quality before committing, the unreliable and the reliable become hard to tell apart, to the disadvantage of both the searcher and the genuinely good site. An edited directory works against that confusion, because inclusion in it carries, however imperfectly, the judgement of an editor who looked. The directory was therefore built to do two things at once: to lower the cost of search, and to act as a filter for quality.
A short history: from Yahoo’s catalogue to the Open Directory Project
The form is best understood through the two directories that defined it. What became Yahoo began, in 1994, as a hand-built catalogue of websites — a list its founders maintained and sorted by category — and for several years a directory of this kind was a, perhaps the, principal way that ordinary users found their way around the web. Inclusion was decided by editors, which made the catalogue trustworthy and also made it a bottleneck: as the web grew, the editors could not keep pace, and being listed became difficult enough to draw complaint.
It was partly in answer to that bottleneck that the Open Directory Project, almost always known as DMOZ, was founded in 1998. Its method was the same in principle — websites placed by human editors into a subject hierarchy — but its labour was distributed across a large body of volunteer editors rather than a single company’s staff, and its data was made openly available for other services to use. At its height DMOZ catalogued several million sites across roughly a million categories, maintained by tens of thousands of volunteers, and its classification was reused widely, including by the major search engines of the day.
The same year, 1998, saw the founding of Google, and with it the ascent of a different method, treated at length in the next article of this series: the automated crawling and algorithmic ranking of the whole web rather than the editorial cataloguing of a selected part of it. The shift in user behaviour that followed was decisive. Yahoo moved its directory steadily into the background and closed it at the end of 2014; DMOZ, increasingly little used, closed in March 2017, though a volunteer mirror of its final state has been kept online since under the name Curlie. It would be a mistake, however, to read these closures as the end of the web directory. They were the end of the attempt to catalogue the whole web by hand — a different and more modest claim, as the rest of this article argues.
The link-building era and what it did to directories
No honest account of the web directory can pass over the period in which the form was, to a substantial degree, bent toward an interest that had nothing to do with helping a reader find a website. The distortion followed directly from the way search engines work. Because a search engine of the kind Brin and Page (1998) described treats a link from one site to another as a sort of endorsement, and ranks sites in part by the links pointing at them, a link from any reasonably regarded site came to have a value quite independent of whether a human being would ever click it. A directory listing is, among other things, a link. It did not take long for site owners, and the firms they paid, to observe that appearing in many directories produced many links, and that many links could lift a site’s standing in search results.
Through much of the 2000s this turned a large part of the web-directory ecosystem into an instrument of search-engine optimization rather than of discovery. Directories multiplied, a great many of them created for no purpose other than to sell listings to owners pursuing links; editorial judgement, the very quality that had given the form its worth, was in many of these cases quietly dropped, since the buyer of a link-driven listing did not require it and the seller had no reason to supply it. The outcome was a large population of directories that were directories in form but not in function, and the genuine edited directories suffered by association with them. The episode closed when the search engines answered it. Across a sequence of changes to their ranking systems they became steadily better at identifying and discounting links that existed only to manipulate rank, and links from low-quality directories lost first much and then most of their value. The lesson worth carrying away is not that directories were discredited as such, but that a directory built for the wrong reader — for a ranking algorithm rather than for a person — was always a hollow thing, and that the directories which kept their worth through the period were precisely those that had never stopped being built for human readers in the first place.
What a web directory is for in 2026
Discovery by browsing rather than searching
A search engine serves the user who can already name what they want; a directory serves the user who cannot. The two states of mind are genuinely different. A person who types a precise query has, in effect, already done the work of knowing what to ask for, whereas a person browsing a category is still forming the question, and a well-built hierarchy supports that earlier and vaguer stage of enquiry by showing what exists. The value of browsing did not disappear when general search improved; it simply ceased to be the way most people navigate the whole web. Within a bounded subject — a single industry, a region, a field of study — moving through a considered set of categories remains a good way to learn the shape of a territory rather than to retrieve one known point in it.
Curation as a signal of quality
The second enduring function is the one introduced above as a filter. A general search engine returns what its ranking system estimates to be relevant, and that estimate, however sophisticated, is an estimate built from signals rather than a judgement that a human being made about the worth of a site. An edited directory offers something the search result does not: the fact that a person assessed the site and chose to include it. For some purposes that fact is worth little, and for others it is worth a great deal — particularly in fields where the cost of relying on a poor source is high, and where, in the terms of Akerlof (1970), a searcher cannot easily tell a sound source from an unsound one without help. A directory’s value here is not breadth, which it cannot match, but the presence of a deliberate editorial decision behind each entry.
A machine-readable map of a domain
There is a third function, less obvious and more particular to 2026. A web directory, because it is structured — because its contents sit in named categories with defined relationships — is a machine-readable map of whatever domain it covers. As search increasingly returns composed answers rather than lists of links, and as automated assistants take on more of the early work of finding things, systems of that kind rely on exactly this sort of structured, categorized, corroborated data. A directory that is well maintained, with accurate categories and verified entries, is a source such systems can use with some confidence. This is industry observation rather than settled research, and should be weighed as such; but it points to a real shift in where a directory’s value lies. The value is migrating from the directory as a place a person browses toward the directory as a structured description of a domain that both people and machines can rely upon.
Why the niche directory is the form that survived
The three functions just described — browsing, curation, and a structured map — do not survive equally well in every kind of directory, and it is the niche directory, introduced earlier, in which they survive best. The reason returns once more to scale. A general directory of the whole web fails because the territory it must cover grows without limit while its editors do not; a niche directory escapes that particular failure because its territory is finite, and a finite subject can be catalogued thoroughly and then kept current by a manageable quantity of work. Within a single field, the editorial judgement that filters for quality remains affordable, the hierarchy stays small enough to browse without exhaustion, and the structured map stays complete enough to be relied upon. A directory of this kind is also, in economic terms, a two-sided platform of the sort analysed by Rochet and Tirole (2003) and by Hagiu and Wright (2015): it sits between the sites that wish to be found and the readers who wish to find them, and it grows more useful to each of those groups as it succeeds in attracting the other. That structure gives a focused directory a durable reason to exist, so long as it genuinely serves a real audience well rather than merely existing for its own sake. The general catalogue of everything was an ambitious attempt that the sheer size of the web defeated; the directory that accepts a boundary and works within it is the form that has a future, and it is where a reader looking for a living example of the web directory should now expect to find one.
Concluding remarks
A web directory, then, is a catalogue of websites organized by subject and assembled, at least in part, by human judgement, built to lower the cost of finding things online and to act as a filter for quality. It is not a search engine, because it is browsed rather than queried and is bounded rather than comprehensive, and it is not a business directory, because its subject is websites rather than organizations. The great hand-built catalogues of the whole web — Yahoo’s directory, the Open Directory Project — have closed, and it is easy to mistake their closure for the end of the form. The more accurate reading is that the ambition to catalogue everything by hand proved unsustainable, while the underlying functions of a directory — structured browsing, editorial curation, a clean map of a domain — did not. Those functions survive, most healthily, in directories that have given up breadth in exchange for depth within a defined subject.
Future developments
The likely direction of change follows from the third function described above. As the systems that answer questions become less like indexes and more like synthesizers, the premium on clean, structured, verifiable descriptions of a domain will rise, and a web directory is, in essence, such a description. The form most likely to matter in the coming years is therefore not the general directory attempting to cover the whole web, which the history above suggests cannot be sustained, but the focused directory that covers one subject well and keeps its categories and entries accurate. Its readership may increasingly be machines as much as people. If that proves correct, the defining work of a web directory will be the same unglamorous discipline it always was — deciding carefully what to include, describing it accurately, and keeping it current — valued now because a structured and trustworthy map of a domain is exactly what an automated answer is built from.
Table 1. Notable general web directories
| Directory | Began | Status | Editorial model |
|---|---|---|---|
| Yahoo Directory | 1994 | Closed 2014 | Editors employed by the company |
| Open Directory Project (DMOZ) | 1998 | Closed 2017 | Distributed volunteer editors; openly licensed data |
| Curlie | 2017 | Active | Volunteer-maintained successor to DMOZ |
| Subject and niche directories | Various | Active | Editorial curation within a single field or region |
References
Akerlof, G. A. (1970). The market for “lemons”: Quality uncertainty and the market mechanism. The Quarterly Journal of Economics, 84(3), 488–500.
Bakos, J. Y. (1997). Reducing buyer search costs: Implications for electronic marketplaces. Management Science, 43(12), 1676–1692.
Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A., & Wiener, J. (2000). Graph structure in the web. Computer Networks, 33(1–6), 309–320.
Brin, S., & Page, L. (1998). The anatomy of a large-scale hypertextual web search engine. Computer Networks and ISDN Systems, 30(1–7), 107–117.
Hagiu, A., & Wright, J. (2015). Multi-sided platforms. International Journal of Industrial Organization, 43, 162–174.
Rochet, J.-C., & Tirole, J. (2003). Platform competition in two-sided markets. Journal of the European Economic Association, 1(4), 990–1029.
Stigler, G. J. (1961). The economics of information. Journal of Political Economy, 69(3), 213–225.
