TL;DR #
- Launching Cloud Shell Connection
- Local Connection
- Enter boot manager
- Enter “edit mode” on grub
- Edit the grub entry
- SELinux (if enabled)
- Remount the root partition
- Change password or add keys
- Back in business
Long version #
In cloud compute instances, the default connection method is SSH, but sometimes something could prevent SSH(boot did not finish, missing key, etc.). Some people argue that troubleshooting some issues is not worth it, create a new instance and move forward. Well, I’m not here to open that can of worms, and I will assume you need to troubleshoot for whatever reason that is important to you. Use with parsimony :)
When the server is on traditional infrastructure, you can plug a KVM(keyboard-video-mouse), either physically or remotely. On OCI, you have a similar option: the Instance Console connection. It is a serial console you can access through SSH or Cloud Shell.
Launching Cloud Shell Connection #
In the instance details, there will be the Console connection under Resources. Clicking on it, there are the choices to *Launch Cloud Shell connection or Create local connection. The first option will do all your work and show the console using Cloud Shell. Using the second option, you need to provide SSH keys, copy the SSH command, run from your SSH client or terminal of choice, and then you will be able to see the console screen. You can see how that is done in the animations below.
Local Connection #
After that, you will be at the login prompt of the console. By default, the opc/ubuntu users do not have passwords set, so at this point, it will not be possible to advance in the process. The choice left is to start the instance in maintenance mode.
Enter boot manager #
First, you will reboot the instance from the console and quickly return to the shell console. Click on the window to set the focus, and press the Esc key repeatedly until you see the following image. Using the arrows, select Boot Manager.
If you see the boot process happening, you missed the chance and will need to reboot the instance and try again.
In the Boot Manager menu, select UEFI Oracle BlockVolume, press Enter, and Esc right after it. It can be tricky on Ubuntu. Some users reported having to use a terminal macro that presses Esc multiple in a timely manner to be able to do this step.
Enter “edit mode” on grub #
In the boot menu, highlight the top item in the menu, and press e to edit the boot entry.
Edit the grub entry #
Once in edit mode, use the down arrow key to scroll down through the entries until you reach the line that starts with linux or linuxefi.
Use the right arrow to move to the end of the line (the End key should take you there too). If you make any mistake, you can press Esc to return to the previous screen and start over by pressing e again. At the end of the line, add:
init=/bin/bash
Press Ctrl + x
Once the boot finishes, the prompt will look like this:
SELinux #
Load the SELinux policies to preserve the context of the files you are modifying:
/usr/sbin/load_policy -i
Remount the root partition #
Remount the root partition with read/write permissions:
/bin/mount -o remount, rw /
Change password or add keys #
At this point, you will be ready to change configurations, manipulate authorized_key files to include/remove keys, set a password for opc/ubuntu user, or perform any other maintenance operation you could not do before because you could not connect using SSH to the instance.
Back in business #
When you are finished, you can either start the system by using:
exec /usr/lib/systemd/systemd
Or reboot the server using:
/usr/sbin/reboot -f
Happy troubleshooting, and let me know how this helped you.