Generating server report from ILO interface (HP servers)

I should use this blog more. Here is one of the things might be useful for other admins I wrote meantime.

This script generates a report, which includes all important information via ILO interface on HP servers (using python-hpilo package). Supports ILO2-3-4, which is a pretty good range for non-ancient servers I think.

I’ve started to study golang as hobby, so might port this to Go later. But for now, this is really handy:

P.s.: There are some disgusting if conditions in script, thanks to ILO. It was insisting to provide inconsistent answers based on version.


Don’t kill the poor process

User gets a warning about disk, sees it’s full and starts to investigate. After removing few GB’s, there is a little problem: df and du commands’ conflicting output.

Where is my free disk?

After searching it on Google, people says “check out lsof | grep deleted” and user realizes this is really a thing on Linux. Processes hold files on disk even the files are deleted. Restarts the process, and everything seems fine now.

But why?

Doing something without understanding first, grinds my gears. I’d never let the soul who chmod 777’s root recursively to touch the keyboard again. I understand this case is not that serious but after a bit digging in the /proc, I’ve found a way to rescue the process’ life!

– Find the process’ PID which keeps the files for itself
– Go to /proc/<PID>/fd
– Run ls -la | grep deleted and find the deleted file’s link (let’s say “13”)
– Redirect something to the link: echo "ok" >| 13
– Check lsof again, you won’t need to get blood on your hands anymore.


Using HP SUM to upgrade firmware

If you have lots of physical servers, I totally understand if you don’t want to talk about “upgrading” them. But you need to upgrade, at least the device firmware’s should be decent so you don’t see weird errors on higher levels. Well ok, you probably will see some weird things anyway but if you upgrade, the vendor won’t be poking you with “please upgrade to latest firmware” for every problem.

This post is about HP servers. There is a tool for firmware management by HP, SUM (Smart Update Manager). You can find the download from google (otherwise it’s not possible, because HP has the flagship of “Hardware vendors must have a terrible website” rule). You can install provided RPM or run it from a Windows box. You should consider a server which has some way to access all others (preferably ssh).

After you run hpsum with no parameters, it’ll prompt the port it’s running (63002), you can access it via browser. Login with the credentials from installed server, don’t search for a default password.


First you should add a “baseline” from baseline library section. Just extract the latest SPP medium HP provided on somewhere and point it as baseline. SUM will scan and detect applicable packages inside.
Then you can add nodes you want to check/upgrade. SUM can connect directly to OS, which is my preferred method. You can of course connect it to ILO, in this case it’ll only check/upgrade the ILO version.

After connection, you should run “inventory” to see current situation on the server. An inventory report will come and you will be selecting components you’d want to be upgraded. Be sure to check reboot options section. You can reboot right after upgrade (if needed) or wait for a more appropriate time and do it manually.


A side note: Run the inventory again after you’ve upgrade/reboot. Sometimes there might be some “middle steps” between firmware versions.

If you like to be quick or want to perform some bulk actions (like adding/deleting nodes/baselines), just use parameters for hpsum. It’ll make your life easier:

Edit: Of course you can see this software crash after a period of frequent usage (maybe also a HP standard). If you see nonsense behaviour, just clear everything with “clean-cache.{sh,bat}” from installation directory.


Install X (software) to Y (distro) [rant]


In this rant, we’ll install X software to Y system. You are here because you are a bit desperate finding the right package you need. Maybe the software and/or the system is a bit old.

Let’s start!

First, download the software here: software.tar.gz


Yes you saw damn right, a big blue TARBALL. You might not care about your system’s file integrity and be a moron to not to think about that COMPILING thing!

For flip’s sake.. You can compile ffmpeg on a potato these days. Don’t send posts like this one or better, package the damn thing for the Y 🙂

Have a nice day!


Why do we use a single distro?

New distros with cool ideas always amaze me. Too bad we generally have new shiny debuntu based software collections only.

Bedrock is a different one. Here is a quote from official website:

If one would like a rock-solid stable base (for example, from Debian or a RHEL clone) yet still have easy access to cutting-edge packages (from, say, Arch Linux), automate compiling packages with Gentoo’s portage, and ensure that software aimed only for the ever popular Ubuntu will run smoothly – all at the same time, in the same distribution – Bedrock Linux will provide a means to achieve this.


That is a really cool idea, having your cake and eating it too! With Bedrock, you can use a stable distro like Debian and use applications from AUR, Portage or whatever you want. It’s still at beta stage, but it took me thinking “How the hell?” so I’ve fired a virtual machine and installed a Debian Jessie to “hijack” and installed ArchLinux as a “strata”. You can enable/disable and switch to a strata with a single command. If your system doesn’t have an application, you can still run it if it’s installed in an installed strata. Here is a poor-sound demo.

If you’re an old distro-hopper like me, you should totally check it out. I don’t remember liking an idea like this since HadronLinux.