# Troubleshooting

## Boot problems

If NixOS fails to boot, there are a number of kernel command line parameters that may help you to identify or fix the issue. You can add these parameters in the GRUB boot menu by pressing “e” to modify the selected boot entry and editing the line starting with `linux`. The following are some useful kernel command line parameters that are recognised by the NixOS boot scripts or by systemd:

`boot.shell_on_fail`

  Start a root shell if something goes wrong in stage 1 of the boot process (the initial ramdisk). This is disabled by default because there is no authentication for the root shell.

`boot.debug1`

  Start an interactive shell in stage 1 before anything useful has been done. That is, no modules have been loaded and no file systems have been mounted, except for `/proc` and `/sys`.

`boot.trace`

  Print every shell command executed by the stage 1 and 2 boot scripts.

`single`

  Boot into rescue mode (a.k.a. single user mode). This will cause systemd to start nothing but the unit `rescue.target`, which runs `sulogin` to prompt for the root password and start a root login shell. Exiting the shell causes the system to continue with the normal boot process.

`systemd.log_level=debug systemd.log_target=console`

  Make systemd very verbose and send log messages to the console instead of the journal.

For more parameters recognised by systemd, see systemd 1.

If no login prompts or X11 login screens appear (e.g. due to hanging dependencies), you can press Alt+ArrowUp. If you’re lucky, this will start rescue mode (described above). (Also note that since most units have a 90-second timeout before systemd gives up on them, the `agetty` login prompts should appear eventually unless something is very wrong.)

## Maintenance mode

You can enter rescue mode by running:

    $ systemctl rescue

This will eventually give you a single-user root shell. Systemd will stop (almost) all system services. To get out of maintenance mode, just exit from the rescue shell.

## Rolling back configuration changes

After running `nixos-rebuild` to switch to a new configuration, you may find that the new configuration doesn’t work very well. In that case, there are several ways to return to a previous configuration.

First, the GRUB boot manager allows you to boot into any previous configuration that hasn’t been garbage-collected. These configurations can be found under the GRUB submenu “NixOS - All configurations”. This is especially useful if the new configuration fails to boot. After the system has booted, you can make the selected configuration the default for subsequent boots:

    $ /run/current-system/bin/switch-to-configuration boot

Second, you can switch to the previous configuration in a running system:

    $ nixos-rebuild switch --rollback

This is equivalent to running:

    $ /nix/var/nix/profiles/system-N-link/bin/switch-to-configuration switch

where N is the number of the NixOS system configuration. To get a list of the available configurations, do:

    $ ls -l /nix/var/nix/profiles/system-*-link
    ...
    lrwxrwxrwx 1 root root 78 Aug 12 13:54 /nix/var/nix/profiles/system-268-link -> /nix/store/202b...-nixos-13.07pre4932_5a676e4-4be1055

## Nix store corruption

After a system crash, it’s possible for files in the Nix store to become corrupted. (For instance, the Ext4 file system has the tendency to replace un-synced files with zero bytes.) NixOS tries hard to prevent this from happening: it performs a `sync` before switching to a new configuration, and Nix’s database is fully transactional. If corruption still occurs, you may be able to fix it automatically.

If the corruption is in a path in the closure of the NixOS system configuration, you can fix it by doing

    $ nixos-rebuild switch --repair

This will cause Nix to check every path in the closure, and if its cryptographic hash differs from the hash recorded in Nix’s database, the path is rebuilt or redownloaded.

You can also scan the entire Nix store for corrupt paths:

    $ nix-store --verify --check-contents --repair

Any corrupt paths will be redownloaded if they’re available in a binary cache; otherwise, they cannot be repaired.

## Nix network issues

Nix uses a so-called binary cache to optimise building a package from source into downloading it as a pre-built binary. That is, whenever a command like `nixos-rebuild` needs a path in the Nix store, Nix will try to download that path from the Internet rather than build it from source. The default binary cache is `http://cache.nixos.org/`. If this cache is unreachable, Nix operations may take a long time due to HTTP connection timeouts. You can disable the use of the binary cache by adding `--option use-binary-caches false`, e.g.

    $ nixos-rebuild switch --option use-binary-caches false

If you have an alternative binary cache at your disposal, you can use it instead:

    $ nixos-rebuild switch --option binary-caches http://my-cache.example.org/