Linux

One day on my main Arch installation I created a container inside a directory, and "booted" into it by using systemd-nspawn. When I was done with it I decided to do a rm -rf / inside the container just to be funny. Then I noticed that my DE on the host froze and I couldn't do anything. Then I realized that systemd-nspawn mounts some important host's directories on the container, and I deleted those when I did the rm -rf /. I didn't lose anything, but it was scary.

Years ago a friend mistakenly typed in killall5 as root on a remote server. Didn't break things but resulted in extra work and effort.

When Ubuntu 16.04 had just been released, I tried upgrading my 14.04, the whole system broke and I had to install another os (Manjaro won).

That day I learned Ubuntu too can be a bit stupid.

So I am sort of an embedded developer, and I like to mess around with weird configurations. So the craziest experiment I did was trying to reflash a rasberry pi from a system running in the pi's RAM. It honestly might have worked, but during the prep work I forgot to resize the filesystem before mucking with the paritions and had to reflash the normal way before I could try again. Ended up just turning it into a pihole instead, but I still learned a lot about pivot_root

One time I rebooted. The system never recovered.

ran chown -R www-data: ./ from /var instead of /var/www.

I've literally done the rm -rf / thing. I thought I was in a different subdirectory, but I was in / and did rm -rf .

When it didn't return after half a second, I looked at the command again and hit CTRL+C about 20 times in the span of 3 seconds.

I had to rebuild the install, but luckily didn't lose anything in /home.

Built a new desktop, backed up everything on my old laptop, next step was to format an Arch installer USB. Instead of formatting the USB, I formatted my laptop's /boot partition. No big loss since I had the backup and was done with that old toaster, but oops.

Ubuntu GUI/apt fail

Back when I used ubuntu, Unity was stuck with old gnome packages. This meant that the version gnome-terminal packaged with ubuntu (up to at least 18.04) didn't have text reflow on window size changes.

You could add the upstream sources, upgrade the specific text reflow package only, and then disable the sources.

I forgot to disable the sources, or typed dist-upgrade (this happened multiple times...). Broke the whole desktop/lightdm setup with half upgraded packages, and half removed packages (for preparation to install new versions). Way easier to reinstall the os than to disentangle. Unity was a mess then anyway.

Moral: Actually read the package change summaries when doing updates/removes/installs, and [ y/N ] means actually check what the fuck you think you're agreeing to.

BtrFS snapshots for idiots

I've also run automated snapshots on my btrfs partition, then run out of space doing multi-hop system upgrade on fedora (dnf has a plugin that creates a snapshot every time it kicks in.

You can imagine there were many changes happenning per snapshot, and I effectively could have rolled back 4 major fedora versions... Til I ran out of space.

I couldn't get a replacement drive in time, and I had an hour to rebuild my laptop before needing to be on a customer site, so sadly I couldn't preserve my drive for later investigation. My best guess is the high-water-mark was configured incorrectly, and somehow it was able to 'write' data past the extents of the filesystem.

Rollback did work for my home partition, but I had to mount it from another OS to get it to work - so no data loss!

By that time I'd already reinstalled the os to the root partition/subvolume however, so I couldn't determine the exact cause of failure :(

Moral: Snapshots are not backups, and 'working' is not 'tested'

sudo usermod -a cdrom

Forgot the -G and wasn't sudo anymore...

I did recover eventually, but it was not nice.

@fl42v I have thousands from my early days, but my only recent-ish one was pretty funny.

On an Arch install that hadn't been updated for a while, in a rush, had an app that needed OpenSSL 3. Instead of updating the whole system, I just updated the openssl package.

*Everything* broke immediately. Turns out a lot of stuff depends on openssl. Who knew?

To fix, booted to the arch installer, chrooted into my env, and reverted to the previous version of the package — then updated properly.

I ran firejail config or something, which replaces a lot of home directory app files. Not sure if binaries or desktop entries.

But things broke, randomly, screenshots not working, not even inside firefox etc. I reinstalled the system and imported the home folder... and it was there again!

I was new to Linux, I made the not so calculated decision to use manjaro as my daily, deleted xorg to in an attempt to reinstall xorg to then hopefully fix the stuttering. Everything went wrong, no display obviously, /boot/ files where corrupt. I now use arch and am wiser

I once did an apt-get upgrade in the middle of when debian testing was recompiling all packages and moving to a new gcc version. I get it, using testing invites stuff like this. But come on, there should at least be a way to warn people beforehand.

I set up a progressive backup of my home folder... to my home folder. By the time I got home that day it was impossible to log in because there was no room to create a login record. Had to fix that by deleting the backup file using a live CD.

I was trying to extract some files from a a Linux image of one of those ARM boards. It was packed into the cpio format, and I had never used the format before. Of course I was trying to extract to a root owned directory and I sudo'ed it. I effed up the command and overwrote all my system directories (/bin, /usr, /lib, etc...). Thankfully I had backed up my system recently and was able to get it working again.

sudo rm -r /run/timeshift

Accidentally executed a JPEG (on an NTFS partition) and the shell started going crazy. reboot was not successful =[

I cant remember anymore... Let me explain ... My first computer was with at-the-time-very-new windows xp, using primary for games, after some time it got bloated with stuff so i had to reinstall again and again over time. Then i discovered redhat,centos and debian... I started heavily distro hopping. My passion for software grew to the point that I was installing new software on daily basis, just to explore new things. But nothing seemed stable enough, ubuntu, fedora, sabayon, gentoo, arch... And their derivatives all broke under my fingers to the point that i had to do more fixing than discovering new software, I took it as a challenge and continue. At around the time of university I discovered NixOS, as with any new technology I went head on with it. It took a lot of trial and error since at the time there were no documentation for any of it. I spent months reading the code, but I never gave up, since what I have found was a gem. I found the OS that is resistant to my curiosity, I just cant seem to be able to break it. Now I use NixOS everywhere that I can, even on my work computer. I do not need to reinstall after initial installation. Well... only when hardware fails...

I had issues with a new version of glibc that prevented me from working on music in Ardour on Manjaro. I then proceeded to force-downgrade glibc (in the hopes of letting me get back to work) and that broke sudo and some other things, which I found out after rebooting. That was an interesting learning experience. Now I snapshot before I do stupid stuff. :]

Once I succumbed to a proprietary software's allure, post-usage, I felt like a digital pariah! To rid myself of the taint, I wiped my system clean – reinstall time!

It was only in a container on a Chromebook, but I'll share it anyway. One time, I had installed Android Studio but found it mildly annoying that I got a line when using apt about Android Studio and some error on a certain line of this one file. I believe the file was something related to dpkg, and after changing some things within the file, I seemed to have broken apt. Luckily, I had a backup, but it was a few days old, so I had to reinstall some apps.

I've not broken my setup (yet), but I've came close to it one time when I accidentally made a lesser fork bomb.

I was writting a function that would display how many jobs I currently had in the command prompt, but when writting the function instead of calling jobs I called the function itself, sourced .bashrc and now everything was laggy (my pc only has 4GB). Thankfully I was able to shut down the terminal before my swap got completely consumed.

I used to work at this place that had a gigantic QNX install. I don't know if QNX that we used back then had any relation to q&x now They certainly don't look very close.

It was in the '90s and they had it set up so that particular nodes handled particular jobs. One node to handle boot images and serve as a net boot provider, one node handled all of the arcnet to ethernet communication, one node handled all the serial to mainframe, a number of the nodes were main worker nodes that collected data and operated machinery and diverters. All of these primary systems were on upper-end 386s or 486s ,they all had local hard disks.

The last class of node they called slave nodes. They were mainly designed for user data ingest, data scanning stations, touch screen terminals, simple things that weren't very high priority.

These nodes could have hard discs in them, and if they did, they would attempt to boot from them saving the net boot server a few cycles.

If for some reason they were unable to boot from their local hard drive, They would netboot format their local hard drive and rewrite their local file system.

If they were on able to rewrite their local file system they could still operate perfectly fine purely off the net boot. The Achilles heel of the system was that you had no idea that they had net booted unless you looked into the log files. If you boot it off your local hard drive of course your root file system would be on your local hard drive. If you had net booted, and it could not rebuild your local file system, your local root file / was actually the literal partition on the boot server. Because of the design of the network boot, nothing looked like it was remotely mounted.

SOP for problems on one of the slave nodes was to wipe the hard disk and reboot, in the process it would format the hard drive and either fix itself or show up as unreliable and you could then replace the disc or just leave the disc out of it. Of course If the local disk had failed and the box had already rebooted off netboot without a technician standing there to witness it, rm -Rf would wipe out the master boot node.

I wasn't the one that wiped it, but I fully understand why the guy did.

Turns out we were on a really old version of QNX, we were kind of a remote warehouse mostly automated. They just shut us down for about a week. Flew a team out. Rebuilt the system from newer software, and setup backups.

Back when I started using Linux, I really wanted something that was super different from windows (I used Gnome 3 for like 3 years). I decided one day to try out Fedora cause, hey, I can live on the bleeding edge.

Second day I had it installed, I was having issues with the audio. Decided to try reinstalling pulse. Apt autoremoved it and somehow completely nuked the entire GUI. Stuck in terminal mode, I found that I had no ethernet to connect to, nor could I figure out how to connect to a wifi network with a password or download packages to a USB. After a couple hours, I gave up, wiped the drive, and went back to Mint.

Nowadays I'm happier in my little comfort zone.

I wanted my top bar in DWM toshow the time, so I put the script directly into the .xinitrc file instead of the path to the script.

About a year ago I somehow fucked up installing a new window manager on my tablet so badly I had to start from scratch - to this day I have no idea what happened there, but it just wouldn't boot properly or anything after that 🤷 I needed it for school pretty quickly though so my top priority was getting it working again, so I set up a fresh install instead of continuing to fuck around.

Not the same level of destruction, but I fucked up my first ever install a couple months in trying to resolve dependencies related to python and wine, which is why I'm more interested in sandboxing whenever feasible these days. After only two months I guess I had been fucking around with linux long enough to have a little too much unearned confidence, lol

I've had the typical disasters with partition tables and boot loader mixups, but the one I keep coming back to is updating my Nvidia drivers too eagerly. Whether something gets messed up with an external monitor, or the laptop starts resisting switching away from the integrated GPU, or an electron app I use regularly that makes heavy use of 3D acceleration breaks, or I just need to bump the driver version in a reproducible system state record... it's just bad news.