Crawler lists track database research progress. That is the plain answer, and it is the part worth keeping in view. A crawler list is a working list of sites, records, or targets that a crawler is meant to visit and keep track of as the work moves ahead.
I use that phrase with care, because “crawler list” can mean a few things in practice. In database work, it often means the seed list or target list that starts a crawl. In platform settings, it can also mean a list of crawler jobs, crawler names, or crawl runs that show what has already been done and what still needs attention.
That is why the list matters. It is not just a set of names on a screen. It is a record of scope. It tells me what the crawl was meant to cover, what was found, and where the process stopped or needs another pass.
For research teaching, that is the useful part. A crawler list helps people see research progress in a way that feels concrete. The list gives a trail of work. It can show growth over time, missing areas, and the difference between planned coverage and actual coverage.
I like that this kind of list makes hidden labor visible. People often talk about a database as if it simply exists. It does not. Someone set the targets, ran the crawl, reviewed the results, and decided what counts as finished enough for now. A crawler list is one place where that work can be seen.
The best use of the list is not to pretend it is the full truth. It is a progress tool, not a final map. If a source is absent, that may mean the crawler missed it, could not reach it, or has not been run on it yet. The list can point to the gap, but it does not always explain the cause.
That limit matters. A crawler list can look complete when it is not. It can also look thin when the real work sits behind it in rules, filters, and crawl logs that are not easy to read. So I treat the list as evidence of process first and evidence of completeness second.
This is the part I want readers to hold onto. A crawler list tracks database research progress by showing what has been targeted, what has been collected, and what remains open. It is a simple tool, but it carries real meaning for anyone trying to judge a database or a digital collection with care.
It also helps with teaching because it shifts the question from “Is this database finished?” to “What stage is this work in?” That is a better question. It makes room for uncertainty, and it gives students a way to ask better follow-up questions about coverage, timing, and method.
Still, one honest limit stays in place. A crawler list does not tell us everything about quality. It does not fully show why some items are missing, how the crawler ranked sources, or whether the list itself was built from a narrow starting set. Those details often sit outside the list, and they can shape the result as much as the crawl itself.
So I read crawler lists as working documents. They are useful because they show motion. They are limited because they do not explain every choice behind that motion. That balance is the real lesson, and it is one reason I keep returning to plain language when I talk about database tools.
The Source List keeps that same promise in small form: one digital source worth knowing, one search tip, and one honest limitation. That is enough for this kind of work, because the point is not to dress up the tool. The point is to make the work behind the database easier to see.