Linux Systems Administration
Installation, configuration, systemd, journalctl, performance analysis, troubleshooting, automation and the investigation of suspiciously unhelpful error messages.
A message from somewhere just around the corner
Well, this is
rather curious.
Welcome to WorldHQ: specialist Linux and Proxmox consultancy for peculiar servers, mysterious outages, temperamental clusters, and other things that ought to know better.
Linux · Proxmox · Unusual Incidents
Our Proxmox cluster is easier to find on Tuesdays.
Nobody knows why.
RAID IS NOT A BACKUP ✳
HAVE YOU CHECKED THE LOGS? ✳
PROXMOX IS NOT WITCHCRAFT. USUALLY.
What happens here?
We troubleshoot
the slightly impossible.
WorldHQ provides practical Linux and Proxmox consultancy for systems that need a little expert attention. From a stubborn service to a virtualisation cluster with delusions of grandeur, we investigate the evidence, explain the problem, and help put things right. We cannot explain everything. We have tried. The paperwork was astonishing.
Proxmox & Virtualisation
Proxmox VE, virtual machines, LXC containers, resource allocation, storage integration and the management of virtual worlds that occasionally forget their place.
Storage & Filesystems
LVM, RAID, filesystems, capacity planning, disk replacement, encryption with LUKS and storage layouts designed to survive more than a particularly awkward Tuesday.
Clustering & High Availability
Proxmox clusters, quorum, cluster communications, high availability and fault investigation. We investigate why every node claims to be healthy while everything is on fire.
Networking & Security
Linux networking, SSH, routing, DNS, firewalls, secure remote access and practical security hardening. We prefer a well documented firewall to a well documented incident.
Backups & Recovery
Backup strategy, Proxmox Backup Server, recovery planning, restoration testing and the gentle reminder that redundancy is not the same thing as a backup. Neither is optimism.
The people in the building
Entirely real.
On paper.
Our staff are experienced, discreet, and almost certainly not a collection of owls operating a Linux server from inside a particularly convincing coat. WorldHQ does not comment on personnel rumours.
Marigold Finch
Director of Unusual Incidents
Can identify a suspicious silence at forty paces. Keeps emergency biscuits in a locked drawer, and the key in another locked drawer. Insists every outage has a cause, preferably recorded in the journal.
Prof. Basil Quibble
Storage & Virtualisation
Can explain a storage layout to a startled dragon. Specialises in virtualisation and the mysterious disappearance of available disk space. Maintains that RAID is not a backup, even when asked very nicely.
Captain Pru Button
Networks & Security
Has never met a daemon she couldn't trace. Once secured a network so thoroughly that even DNS needed a written appointment. Takes SSH security very seriously indeed.
John Hodson
Grand Keeper of the Root Password · Principal Linux & Proxmox Specialist
WorldHQ's resident Linux and Proxmox specialist, John
deals with kernels, filesystems, networking, security,
storage, virtualisation and clusters that have developed
an unfortunate relationship with reality.
His particular expertise includes Proxmox VE,
virtual machines, LXC containers, clustering,
high availability and backup strategy.
He believes that almost every technical mystery has
a logical explanation — although occasionally it takes
three terminals, seventeen log files and a cup of tea
to find it.
Small print: For Linux and Proxmox consultancy, system diagnostics, or emergencies involving inexplicable penguin behaviour, contact John on +44(0)7974 126 368. YouTube Please do not call during a kernel panic unless the kernel has actually panicked.
A note from the archives
The cluster
was fine.
Apparently.
Incident report WHQ-404:
The Proxmox cluster reported that every node was healthy. All virtual machines were accounted for. Storage was available. The network was operational.
Unfortunately, one virtual machine appeared to be running somewhere that did not exist on the cluster configuration.
John was summoned. He requested the logs, examined the cluster state, and made a pot of tea.
The virtual machine was eventually located. The tea was excellent.
Conclusion: Further investigation may be required. Reality remains under observation.
# Extract from the incident terminal
root@proxmox:~# pvecm status
Quorate: Yes
Nodes: 3
Expected votes: 3
root@proxmox:~# systemctl status reality.service
Active: active (running)
Main PID: unknown
Location: classified(ish)
# John has requested a second cup of tea.
✳
End of public notice
Linux is logical.
Until it isn't.
Whether you need help with Linux administration, Proxmox
virtualisation, storage, clustering, high availability,
networking or an outage that defies explanation,
WorldHQ is prepared to investigate.
Keep your systems patched, your backups tested,
your logs intact, and your mysterious glowing objects
at a sensible distance.