I need help!
My instance apparently starts. But I can't connect.
The log displays a disk full message. I increased disk space and still can't connect. Serial port output returns the message below.
Ubuntu 21.04 bancodedados ttyS0 bancodedados login: [ 549.942514] cloud-init[987]: OSError: [Errno 28] No space left on device [ 549.942724] cloud-init[987]: During handling of the above exception, another exception occurred: [ 549.942816] cloud-init[987]: Traceback (most recent call last): [ 549.942875] cloud-init[987]: File "/usr/bin/cloud-init", line 33, in <module> [ 549.942941] cloud-init[987]: sys.exit(load_entry_point('cloud-init==21.4', 'console_scripts', 'cloud-init')()) [ 549.942986] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 927, in main [ 549.943035] cloud-init[987]: retval = util.log_time( [ 549.943078] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2472, in log_time [ 549.943416] cloud-init[987]: ret = func(*args, **kwargs) [ 549.943500] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 684, in status_wrapper [ 549.943849] cloud-init[987]: atomic_helper.write_json(status_path, status) [ 549.943930] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 44, in write_json [ 549.944082] cloud-init[987]: return write_file( [ 549.944133] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 39, in write_file [ 549.944201] cloud-init[987]: raise e [ 549.944250] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 33, in write_file [ 549.944295] cloud-init[987]: tf.close() [ 549.944337] cloud-init[987]: File "/usr/lib/python3.9/tempfile.py", line 639, in close [ 549.944613] cloud-init[987]: self._closer.close() [ 549.944685] cloud-init[987]: File "/usr/lib/python3.9/tempfile.py", line 573, in close [ 549.944976] cloud-init[987]: self.file.close() [ 549.945063] cloud-init[987]: OSError: [Errno 28] No space left on device [ 549.945114] cloud-init[987]: Error in sys.excepthook: [ 549.945166] cloud-init[987]: Traceback (most recent call last): [ 549.945213] cloud-init[987]: File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 155, in apport_excepthook [ 549.945265] cloud-init[987]: pr.write(f) [ 549.945308] cloud-init[987]: File "/usr/lib/python3/dist-packages/problem_report.py", line 382, in write [ 549.945362] cloud-init[987]: file.write(v.replace(b'\n', b'\n ')) [ 549.945535] cloud-init[987]: OSError: [Errno 28] No space left on device [ 549.945587] cloud-init[987]: During handling of the above exception, another exception occurred: [ 549.945629] cloud-init[987]: Traceback (most recent call last): [ 549.945681] cloud-init[987]: File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 155, in apport_excepthook [ 549.945744] cloud-init[987]: pr.write(f) [ 549.945790] cloud-init[987]: OSError: [Errno 28] No space left on device [ 549.945917] cloud-init[987]: Original exception was: [ 549.945959] cloud-init[987]: OSError: [Errno 28] No space left on device [ 549.946001] cloud-init[987]: During handling of the above exception, another exception occurred: [ 549.946045] cloud-init[987]: Traceback (most recent call last): [ 549.946087] cloud-init[987]: File "/usr/bin/cloud-init", line 33, in <module> [ 549.946188] cloud-init[987]: sys.exit(load_entry_point('cloud-init==21.4', 'console_scripts', 'cloud-init')()) [ 549.946244] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 927, in main [ 549.946357] cloud-init[987]: retval = util.log_time( [ 549.946403] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2472, in log_time [ 549.946931] cloud-init[987]: ret = func(*args, **kwargs) [ 549.947011] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 684, in status_wrapper [ 549.947324] cloud-init[987]: atomic_helper.write_json(status_path, status) [ 549.947392] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 44, in write_json [ 549.947452] cloud-init[987]: return write_file( [ 549.947501] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 39, in write_file [ 549.947739] cloud-init[987]: raise e [ 550.248548] cloud-init[987]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 33, in write_file [ 550.248722] cloud-init[987]: tf.close() [ 550.248772] cloud-init[987]: File "/usr/lib/python3.9/tempfile.py", line 639, in close [ 550.248805] cloud-init[987]: self._closer.close() [ 550.248834] cloud-init[987]: File "/usr/lib/python3.9/tempfile.py", line 573, in close [ 550.248863] cloud-init[987]: self.file.close() [ 550.248897] cloud-init[987]: OSError: [Errno 28] No space left on device [ 551.380242] cloud-init[988]: OSError: [Errno 28] No space left on device [ 551.380549] cloud-init[988]: During handling of the above exception, another exception occurred: [ 551.380606] cloud-init[988]: Traceback (most recent call last): [ 551.380651] cloud-init[988]: File "/usr/bin/cloud-init", line 33, in <module> [ 551.380712] cloud-init[988]: sys.exit(load_entry_point('cloud-init==21.4', 'console_scripts', 'cloud-init')()) [ 551.380757] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 927, in main [ 551.380810] cloud-init[988]: retval = util.log_time( [ 551.380856] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2472, in log_time [ 551.380910] cloud-init[988]: ret = func(*args, **kwargs) [ 551.380953] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 684, in status_wrapper [ 551.382313] cloud-init[988]: atomic_helper.write_json(status_path, status) [ 551.382407] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 44, in write_json [ 551.382454] cloud-init[988]: return write_file( [ 551.382510] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 39, in write_file [ 551.382558] cloud-init[988]: raise e [ 551.382603] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 33, in write_file [ 551.382646] cloud-init[988]: tf.close() [ 551.382695] cloud-init[988]: File "/usr/lib/python3.9/tempfile.py", line 639, in close [ 551.382737] cloud-init[988]: self._closer.close() [ 551.382779] cloud-init[988]: File "/usr/lib/python3.9/tempfile.py", line 573, in close [ 551.382821] cloud-init[988]: self.file.close() [ 551.382863] cloud-init[988]: OSError: [Errno 28] No space left on device [ 551.382904] cloud-init[988]: Error in sys.excepthook: [ 551.382946] cloud-init[988]: Traceback (most recent call last): [ 551.382987] cloud-init[988]: File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 155, in apport_excepthook [ 551.383028] cloud-init[988]: pr.write(f) [ 551.383070] cloud-init[988]: File "/usr/lib/python3/dist-packages/problem_report.py", line 382, in write [ 551.383115] cloud-init[988]: file.write(v.replace(b'\n', b'\n ')) [ 551.383159] cloud-init[988]: OSError: [Errno 28] No space left on device [ 551.383203] cloud-init[988]: During handling of the above exception, another exception occurred: [ 551.383381] cloud-init[988]: Traceback (most recent call last): [ 551.383435] cloud-init[988]: File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 155, in apport_excepthook [ 551.383490] cloud-init[988]: pr.write(f) [ 551.383536] cloud-init[988]: OSError: [Errno 28] No space left on device [ 551.383582] cloud-init[988]: Original exception was: [ 551.383625] cloud-init[988]: OSError: [Errno 28] No space left on device [ 551.383667] cloud-init[988]: During handling of the above exception, another exception occurred: [ 551.383715] cloud-init[988]: Traceback (most recent call last): [ 551.383756] cloud-init[988]: File "/usr/bin/cloud-init", line 33, in <module> [ 551.383803] cloud-init[988]: sys.exit(load_entry_point('cloud-init==21.4', 'console_scripts', 'cloud-init')()) [ 551.383853] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 927, in main [ 551.383919] cloud-init[988]: retval = util.log_time( [ 551.383965] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/util.py", line 2472, in log_time [ 551.384005] cloud-init[988]: ret = func(*args, **kwargs) [ 551.384044] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/cmd/main.py", line 684, in status_wrapper [ 551.384088] cloud-init[988]: atomic_helper.write_json(status_path, status) [ 551.384133] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 44, in write_json [ 551.384182] cloud-init[988]: return write_file( [ 551.384226] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 39, in write_file [ 551.727184] cloud-init[988]: raise e [ 551.727254] cloud-init[988]: File "/usr/lib/python3/dist-packages/cloudinit/atomic_helper.py", line 33, in write_file [ 551.727289] cloud-init[988]: tf.close() [ 551.727319] cloud-init[988]: File "/usr/lib/python3.9/tempfile.py", line 639, in close [ 551.727347] cloud-init[988]: self._closer.close() [ 551.727376] cloud-init[988]: File "/usr/lib/python3.9/tempfile.py", line 573, in close [ 551.728142] cloud-init[988]: self.file.close() [ 551.728180] cloud-init[988]: OSError: [Errno 28] No space left on device [ 610.581427] google_guest_agent[558]: 2022/03/11 09:59:40 logging client: context deadline exceeded [ 1210.581652] google_guest_agent[558]: 2022/03/11 10:09:40 logging client: context deadline exceeded [ 1810.581942] google_guest_agent[558]: 2022/03/11 10:19:40 logging client: context deadline exceeded [ 2410.582202] google_guest_agent[558]: 2022/03/11 10:29:40 logging client: context deadline exceeded [ 3010.582585] google_guest_agent[558]: 2022/03/11 10:39:40 logging client: context deadline exceeded [ 3610.582876] google_guest_agent[558]: 2022/03/11 10:49:40 logging client: context deadline exceeded
Solved! Go to Solution.
I resolved the problem in a simple way.
I did use the steps described in the documentation and the system was restored
https://cloud.google.com/compute/docs/disks/recover-vm
This other link may also help
https://cloud.google.com/compute/docs/disks/resize-persistent-disk
You might have to resize the file system manually. If your VM does not support automatic root file system resizing, or if you've resized a data (non-boot) persistent disk, you must manually resize the file system and partitions.
I also recommend you run command "df -i" to check the file system, and clean up "/tmp".
I have the same issue and I cannot even SSH to it.
How will I be able to run those commands?
I believe the solution that remains is to attach the disk to another temp created instance, remove unneeded files and retry attaching then working on the same old machine
I resolved the problem in a simple way.
I did use the steps described in the documentation and the system was restored
https://cloud.google.com/compute/docs/disks/recover-vm
This other link may also help
https://cloud.google.com/compute/docs/disks/resize-persistent-disk