I had a lost+found folder in all Unix file systems I used since the 80s. It's where fsck places files that it found during a scan and can't figure out to which directory they belong. Sometimes I found stuff in there.
From what I googled XFS, Btrfs and ZFS don't use lost+found. It's a thing of the old not journaled filesystems and of the ext family.
XFS does use /lost+found, it calls it the "orphanage directory" and xfs_repair reparents children of corrupt directories there.
Based on comments in the kernel source, it seems like the userspace fsck for JFS and F2FS will also sometimes create /lost+found. There might be more that do.
XFS filesystems do not have a "/lost+found" directory in their normal state.
In the very rare occasions when one has to run "xfs_repair", it will create a "/lost+found" directory, if it is required for recovered files.
After the repair and after investigating whether the recovered files contain useful data or not (and after moving the useful files elsewhere), one should normally delete the "/lost+found" directory, because it is no longer needed.
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.
ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).
I have a book on my bookshelf, Eric Foxley's Unix for Super-Users. It was published in 1985, and it answers this question on page 52, the first page listed for the entry 'lost+found' in its index.
This is surely not the earliest book mention, is it? (It'll be in earlier man pages, of course.) Google Books does not give me an earlier one, although it does yield another 1985 book.
Fun fact: Foxley cautioned that lost+found must be pre-sized ahead of time, because the fsck of the time did not grow the directory to fit found files.
Foxleys book does not actually claim reason why it need to be preallocated. I've got the book also. Index mentions fsck only page 52, where it reads:
"52 UNIX FOR SUPER-USERS
...
The filestore consistency check is performed by the command fsck (usu-
ally stored as /etc/fsck, but sometimes in /bin), which should be used to check
all discs used as file-systems. It defaults to the list of filestore devices given
in the file /etc/checklist. At this stage, most of the file-systems will not be
mounted, so will be inactive; only the root file-system will be active. The
fsck command goes through each one in turn, reports any inconsistencies in
them, and offers to correct them. The reply to each query is either 'y' for yes
(correct the inconsistency), or 'n' for no (leave the file-system inconsistent).
A parameter '-y' to the command assumes 'yes' replies to all questions, so
that no further interaction is necessary; a parameter '-n' similarly assumes
all 'no' answers, and therefore needs no write permission to the device. Any
'yes' reply may involve the loss of information, such as the complete removal
of a suspect file. Suspect files on the file-system being checked are written to
a directory lost + found on the device if such a directory exists; this directory
must have been created, and be sufficiently large already to hold the names
of all the files involved. This can be ensured by first creating the directory,
then creating a number of files in the directory, and then removing them.
The corrected systems will be consistent, and can later be mounted as and
when required. It may be possible to recover information from deleted files
by looking at the lost + found directory. There should be a lost + found direc-
tory at the head of each mountable file-system.
When checking the root file-system, there are complications, in that it
will be active (even though, because it is root, it will not be formally mounted
as such, but is implicitly mounted as root during the booting process). If
modifications are necessary, they should be completed, and the machine
rebooted without first performing a sync (see section 4.5 below for the nor-
mal procedure for taking a system down). This is to ensure that the disc as
modified by fsck is not overwritten by any in-core information, which may
have been generated from information read from the original corrupt (incon-
sistent) version.
...
"
But I've also read more detailed explanation that recollect is that unless blocks were preallocated, there was a possibility that lost+found need first allocate more blocks directory it already had, it would possibly led to losing some data that would otherwise been able to recover once fsck had advanced further from that point.
Old time UNIX systems directories were just another structured file, which a 'd' bit (like others you changed with chmod) on them, where each record was 16 bytes, which 2 first were the inode number followed by 14 bytes reserved for filename. IIRC linux also had first same limit first filesystems. You could read directory with any program, common feat was to check any odd stuff that "ls" would not show with could hexdump or "od" with some flags and print 16 bytes lines per row. That way you also could see any deleted or moved files from that directory, because directory entry was not quite long otherwise cleared but just clearing that inode reference two bytes.
A Quick look from other books that I have close me now Maurice J. Bach, The Design of Operating UNIX System (-86) contains much more about fsck and filesystem fixing issues scattered few pages in the book and what methods were used to mitigate loss of data. S.R Bourne The Unix System, no mention of fsck at all, at least by looking book automatically generated index.
It could have been some other quite old book I did read, but did not own or anything since BSD4.3-tahoe documentation I've read over the years. But sure it would be nice to read that exact reasoning again from credible sources.
edit: Oh, and you could preallocate also just by adding entries or copying some data to lost+found enough, and then remove entries. Unix traditionally have not compacted and resized directories. They only grow and can be if have been very large slow to traverse. The way to compact is creating another, moving existing data there and then swapping directories.
OK, I found bit more from UNIX System Administration Handbook Third Edition (Evi Nemeth, Garth Snyder, Scott Seebass, Trent R. Hein, with even more authors mentioned) 2001 book following:
"134 UNIX System Administration Handbook
...
The lost+found directory is automatically created when you build a filesystem. It is
used by fsck in emergencies; do not delete it. The lost+found directory has some
extra space prealocated so that fsck can store "unlinked" files there without having
to allocate additional directory entries on an unstable filesystem. Some systems pro-
vide mklost+found command that can recreate this special directory if it is aci-
dentally deleted.
...
"
The occassional "Drive has not been checked in <n> days, forcing check" message on bootup got annoying sometimes, yeah. It could easily take tens of minutes to finish, exactly when I wanted to use the computer!
(At least this is what my memory is telling me. I could be mistaken, but that's what I remember.)
I used to develop SSD firmware and one of things I worked on is making it robust to power failure. The power supplies have lots of capacitance so the voltage drop was slow so we would use a special test board that would disconnect from power and discharge fast to test it.
That's what the answers are missing, of course. In some filesystem formats, it's possible either to recover completely from a journal/intent log, or at least to recover everything to the point that recovered files can be placed into the correct directory.
Same here. And I had some pretty f**ed up file systems.
At one point, I had one where the directory structure was completely broken and had circles in it (broken SSD). To be fair, in that particular case, I did not look for lost+found and just wrote a tool to extract the data manually that I was looking for.
The lost+found folder saved our backs once way back when. We had a 1TB NAS running Linux, an enormous amount of storage back then. It was shut down unexpectedly and disorderly and ran a multi hour fsck upon reboot. The volume must have had a shot root inode, as after the machine booted and mounted the volume it was empty. All directories were luckily under lost+found with all their contents.
We once deleted the lost+found folder on an old Unix system* by accident. Things went very badly the next time the system rebooted, fsck did not handle it at all well.
On the UFS and suchlike filesystems, at the point that fsck is rescuing orphaned i-nodes, it still has not fully gone through the process of checking and correcting free list information, or indeed fully eliminating errors from the i-node table. Creating a directory involves allocating a new i-node from an unused slot, and free blocks off the free block list.
Ironically, because they are slightly or grossly different to Unix filesystem formats, on HPFS and FAT this is less of an issue. (FAT usually has unused slots in the root directory that it is sane to use at that point, for example.) CHKDSK on OS/2 did create its \FOUND.nnn files on the fly.
As the submission explains, the lost+found folder has pre-allocated space for the directory entries. From the mklost+found man page [0]:
mklost+found pre-allocates disk blocks to the lost+found directory
so that when e2fsck(8) is being run to recover a file system, it
does not need to allocate blocks in the file system to store a
large number of unlinked files. This ensures that e2fsck will not
have to allocate data blocks in the file system during recovery.
Pre-allocating space without making the directory visible would require more arcane file system magic.
they offer private instances to school too, where moderation is left to school policy, and mine seems to be good enough to use whenever i have frontend questions (i'm 10 year into my career and still use my juniors to answer my frontend questions, i think i won't ever change)
lost+found is still used on OpenBSD, seems it is created when needed. Only /home has that directory on my system. IIRC, it was created when a kernel panic happened a few releases ago. Plus some files were placed in it when fsck executed on /home
Same, it's only in /home on my system also. Also /home is pretty much the only directory where I see fsck needing to do a lot of recovery after a power failure. Makes sense I guess, because that's where processes such as web browsers are likely to have lots of files open in RW mode.
Back in the day I accidentally deleted all my stuff because I had it all in a special dir of this user in suse Linux. When I deleted the user, yast deleted everything.
Fortunately I was using ReiserFS at the time and something about its murderous tree data structure made it trivial to undelete.
Reiser_fsck found ALL my stuff, mostly with full dir tree structure in tact and put it all in lost+found
I've last seen those in older Cisco ASA firewall's up to OS version 8.something which had internal or external CompactFlash with VFAT16 filesystems. Usually caused by end of useful life of that CF or if already replaced by someone who did not quite understand device requirements and what it supports.
Either because did not care or understand 32bit VFAT while it works for while, then when CF usage gets over VFAT16 supported 2GB fs gets corrupted, system fails after a while and then you got plenty of those FOUND.xxx files root of that CF drive after boot ran fsck. Those old ASA's did accept and work with 4GB CF's which were available much longer than 2GB versions, but you needed to make max 2GB VFAT16 primary partition and then format it with Linux mkfs.vfat with flags that made sure it's only 16bit version. Once that was done, you could copy files from old CF using something that copies also hidden files and directories too, which there were few there.
ASA used to be some Cisco proprietary OS and was just rebranded PIX firewall, then from 8 oddly hacked linux with grub boot.
Not much of my favourite box as a firewall, but Remote-access VPN features did work quite well quite long and when clustered it was easy to run upgrades each node at time without DTLS and IPsec clients even noticing it.
Not really, as it's only once per file system mount, whereas those Windows and MacOS files are sprinkled in most directories with images and almost every non-network drive directory respectively.
How do questions like this make it to the top? It is an obvious thing if you search for it or ask AI, but people seem to just ignore those in favor of generating new human responses.
Thing is, any time I try to replicate something like that, I basically get a flippant response saying to go look elsewhere.
I had a lost+found folder in all Unix file systems I used since the 80s. It's where fsck places files that it found during a scan and can't figure out to which directory they belong. Sometimes I found stuff in there.
From what I googled XFS, Btrfs and ZFS don't use lost+found. It's a thing of the old not journaled filesystems and of the ext family.
XFS does use /lost+found, it calls it the "orphanage directory" and xfs_repair reparents children of corrupt directories there.
Based on comments in the kernel source, it seems like the userspace fsck for JFS and F2FS will also sometimes create /lost+found. There might be more that do.
XFS filesystems do not have a "/lost+found" directory in their normal state.
In the very rare occasions when one has to run "xfs_repair", it will create a "/lost+found" directory, if it is required for recovered files.
After the repair and after investigating whether the recovered files contain useful data or not (and after moving the useful files elsewhere), one should normally delete the "/lost+found" directory, because it is no longer needed.
6 replies →
Even with journaling, you might need one. ZeroFS [0] almost had a lost+found directory (even with the WAL enabled), because you might have consistency issues between your in-memory state and what was flushed, and especially in what order.
ZeroFS ended up not needing recovery at all through atomic, strictly ordered commits [1], but it was far from trivial (and not just a matter of requiring a WAL).
[0] https://github.com/Barre/ZeroFS
[1] https://github.com/Barre/ZeroFS/blob/main/zerofs/src/fs/writ...
I have a book on my bookshelf, Eric Foxley's Unix for Super-Users. It was published in 1985, and it answers this question on page 52, the first page listed for the entry 'lost+found' in its index.
This is surely not the earliest book mention, is it? (It'll be in earlier man pages, of course.) Google Books does not give me an earlier one, although it does yield another 1985 book.
Fun fact: Foxley cautioned that lost+found must be pre-sized ahead of time, because the fsck of the time did not grow the directory to fit found files.
How do you size a directory? Just by creating entries in it and then unlinking them?
Usually the filesystem driver provides an API call that sets the appropriate data structures so that the directory entry table is preallocated.
The ext-family filesystems provide the mklost+found command to tap into this call if you need to recreate the lost+found directory specifically.
From memory mklost+found did exactly this
3 replies →
Foxleys book does not actually claim reason why it need to be preallocated. I've got the book also. Index mentions fsck only page 52, where it reads:
But I've also read more detailed explanation that recollect is that unless blocks were preallocated, there was a possibility that lost+found need first allocate more blocks directory it already had, it would possibly led to losing some data that would otherwise been able to recover once fsck had advanced further from that point.
Old time UNIX systems directories were just another structured file, which a 'd' bit (like others you changed with chmod) on them, where each record was 16 bytes, which 2 first were the inode number followed by 14 bytes reserved for filename. IIRC linux also had first same limit first filesystems. You could read directory with any program, common feat was to check any odd stuff that "ls" would not show with could hexdump or "od" with some flags and print 16 bytes lines per row. That way you also could see any deleted or moved files from that directory, because directory entry was not quite long otherwise cleared but just clearing that inode reference two bytes.
A Quick look from other books that I have close me now Maurice J. Bach, The Design of Operating UNIX System (-86) contains much more about fsck and filesystem fixing issues scattered few pages in the book and what methods were used to mitigate loss of data. S.R Bourne The Unix System, no mention of fsck at all, at least by looking book automatically generated index.
It could have been some other quite old book I did read, but did not own or anything since BSD4.3-tahoe documentation I've read over the years. But sure it would be nice to read that exact reasoning again from credible sources.
edit: Oh, and you could preallocate also just by adding entries or copying some data to lost+found enough, and then remove entries. Unix traditionally have not compacted and resized directories. They only grow and can be if have been very large slow to traverse. The way to compact is creating another, moving existing data there and then swapping directories.
OK, I found bit more from UNIX System Administration Handbook Third Edition (Evi Nemeth, Garth Snyder, Scott Seebass, Trent R. Hein, with even more authors mentioned) 2001 book following:
In a couple of decades running Linux installations of all flavours, I have never seen anything in lost+found!
Yea, run an old kernel with ext2 on a busy system writing a bunch of small files and have a power supply fail and you'll end up with something there.
fsck on large hard drives was scary on how long it could take to finish.
The occassional "Drive has not been checked in <n> days, forcing check" message on bootup got annoying sometimes, yeah. It could easily take tens of minutes to finish, exactly when I wanted to use the computer!
(At least this is what my memory is telling me. I could be mistaken, but that's what I remember.)
5 replies →
You need to use worse hardware and bad power :)
I used to develop SSD firmware and one of things I worked on is making it robust to power failure. The power supplies have lots of capacitance so the voltage drop was slow so we would use a special test board that would disconnect from power and discharge fast to test it.
3 replies →
Yeah, I think lack of any decent hardware RAID could be a prerequisite.
And more concurrent writes.
But I think ext4 will only let things appear there if you change some default flags.
Umm .. how about a Raspberry Pi Zero 2W powered by a 2000MaH " lipstick style" powerbank?
That's what the answers are missing, of course. In some filesystem formats, it's possible either to recover completely from a journal/intent log, or at least to recover everything to the point that recovered files can be placed into the correct directory.
Same here. And I had some pretty f**ed up file systems.
At one point, I had one where the directory structure was completely broken and had circles in it (broken SSD). To be fair, in that particular case, I did not look for lost+found and just wrote a tool to extract the data manually that I was looking for.
My SD cards have always had stuff in that folder. It scares me. I try not to look
Have to run fsck. This used to be forced about once a month but don’t remember it happening in the last decade or so.
The lost+found folder saved our backs once way back when. We had a 1TB NAS running Linux, an enormous amount of storage back then. It was shut down unexpectedly and disorderly and ran a multi hour fsck upon reboot. The volume must have had a shot root inode, as after the machine booted and mounted the volume it was empty. All directories were luckily under lost+found with all their contents.
We once deleted the lost+found folder on an old Unix system* by accident. Things went very badly the next time the system rebooted, fsck did not handle it at all well.
* Probably DEC Ultrix 2.2, a BSD 4.2 derivative.
Why can't a filesystem create the lost+found folder only when it needs to store files in it?
That would be a much cleaner approach, imho.
Added benefit is that you'd immediately see it if something is wrong with a disk.
Chicken and egg problem.
On the UFS and suchlike filesystems, at the point that fsck is rescuing orphaned i-nodes, it still has not fully gone through the process of checking and correcting free list information, or indeed fully eliminating errors from the i-node table. Creating a directory involves allocating a new i-node from an unused slot, and free blocks off the free block list.
Ironically, because they are slightly or grossly different to Unix filesystem formats, on HPFS and FAT this is less of an issue. (FAT usually has unused slots in the root directory that it is sane to use at that point, for example.) CHKDSK on OS/2 did create its \FOUND.nnn files on the fly.
As the submission explains, the lost+found folder has pre-allocated space for the directory entries. From the mklost+found man page [0]:
Pre-allocating space without making the directory visible would require more arcane file system magic.
[0] https://man7.org/linux/man-pages/man8/mklost+found.8.html
> Preallocating space without making the directory visible would require more arcane file system magic.
If those filesystem engineers had a manager that said: make this nice for the user, then it would have been done.
But these developers had no managers and were OK eating their own unpalatable dogfood.
2 replies →
Because you need to pre-allocate space and indies for it in case things go wrong and the disk is almost full
(2012), maybe?
It is StackExchange. So in theory someone could modernize it at any time.
Indeed and I just saw it was edited in 2012 - originally posed in 2011, answers last updated in 2014.
1 reply →
glad to see that Stack Overflow (or stackexchange.com) is still a thing.
Won't be for long.
https://news.ycombinator.com/item?id=48282709
It's a question from 2011...
they offer private instances to school too, where moderation is left to school policy, and mine seems to be good enough to use whenever i have frontend questions (i'm 10 year into my career and still use my juniors to answer my frontend questions, i think i won't ever change)
I keep everything hidden there.
In reallife I would rename this to "trash".
lost+found is still used on OpenBSD, seems it is created when needed. Only /home has that directory on my system. IIRC, it was created when a kernel panic happened a few releases ago. Plus some files were placed in it when fsck executed on /home
Same, it's only in /home on my system also. Also /home is pretty much the only directory where I see fsck needing to do a lot of recovery after a power failure. Makes sense I guess, because that's where processes such as web browsers are likely to have lots of files open in RW mode.
Back in the day I accidentally deleted all my stuff because I had it all in a special dir of this user in suse Linux. When I deleted the user, yast deleted everything.
Fortunately I was using ReiserFS at the time and something about its murderous tree data structure made it trivial to undelete.
Reiser_fsck found ALL my stuff, mostly with full dir tree structure in tact and put it all in lost+found
lost+found is the Thumbs.db and .DS_Store of Linux
More like a FOUND.000 folder or a root directory filled with .CHK files
I've last seen those in older Cisco ASA firewall's up to OS version 8.something which had internal or external CompactFlash with VFAT16 filesystems. Usually caused by end of useful life of that CF or if already replaced by someone who did not quite understand device requirements and what it supports.
Either because did not care or understand 32bit VFAT while it works for while, then when CF usage gets over VFAT16 supported 2GB fs gets corrupted, system fails after a while and then you got plenty of those FOUND.xxx files root of that CF drive after boot ran fsck. Those old ASA's did accept and work with 4GB CF's which were available much longer than 2GB versions, but you needed to make max 2GB VFAT16 primary partition and then format it with Linux mkfs.vfat with flags that made sure it's only 16bit version. Once that was done, you could copy files from old CF using something that copies also hidden files and directories too, which there were few there.
ASA used to be some Cisco proprietary OS and was just rebranded PIX firewall, then from 8 oddly hacked linux with grub boot.
Not much of my favourite box as a firewall, but Remote-access VPN features did work quite well quite long and when clustered it was easy to run upgrades each node at time without DTLS and IPsec clients even noticing it.
I was thinking that it’s been a while since I saw those file names… but I guess your username checks out :)
1 reply →
Not really, as it's only once per file system mount, whereas those Windows and MacOS files are sprinkled in most directories with images and almost every non-network drive directory respectively.
I think it’s more akin to the Recovered Items (or something like that) folder that shows up in your home directory sometimes (but maybe not anymore?)
How do questions like this make it to the top? It is an obvious thing if you search for it or ask AI, but people seem to just ignore those in favor of generating new human responses.
Thing is, any time I try to replicate something like that, I basically get a flippant response saying to go look elsewhere.
Reaching HN or other sites allows for exposure to information that a person otherwise would not ask themselves.
I also respect human responses over AI ones every day that ends in Y.
Why not on sundae?