Quickpost: RTC problems in Debian on Dell PowerEdge servers
I’ve had some problems with RTC on Dell PowerEdge servers with Debian 4.0 (etch), which were unfixable by all of the following:
- Set Hz to 100/250/1000
- Try newer kernel
- Add –directisa option to hwclock (Would only fix a symptom of the problem, and not touch the problem itself at all)
The symptoms:
You get a lot of messages like this:
rtc: lost some interrupts at 2048Hz
This varies (in my case) between 1024Hz, 2048Hz and 4096Hz. The message is printed to the terminal as fast as it can handle, and is written to syslog at a rate of 3500/sec – Though some times less.
If you run “hwclock” on the system, it spits out an error message:
select() to /dev/rtc to wait for clock tick timed out.
Chris Snyder suggests to avoid the problem instead of fixing it, by disabling precise timekeeping in VMware (Which is what was my main problem). What did fix the problem, however, was to compile a new kernel with HPET_EMULATE_RTC. This is more or less a battle in itself, as HPET_EMULATE_RTC depends on other stuff on its own. I’m to exhausted to give a proper walkthrough on this, as I’ve done about 50 kernel recompiles in the past week or so, and just want to forget this issue altogether ASAP.
Why HPET_EMULATE_RTC isn’t on by default is beyond my scope of comprehension, as there’s absolutely no good reason to not have it enabled unless you’re running an extremely small embedded kernel which doesn’t need it in the first place, in which case you’ll disabe it forcefully anyway.
Update: After notifying Chris about this blog post, he suggested that I update it with the kernel config options that I enabled to make HPET RTC emulation work. As opposed to what I wrote earlier, there’s not really hundreds of different .config options to edit. Only a few if you’re editing .config directly and only one if you’re using menuconf.
In menuconf, go Device Drivers ---> Character Devices ---> Enchanced Real Time Clock Support. You’ll see that it is built as a module. What we really need is to make it built-in. Just press Y. Now you can type / (slash) and search for HPET. Go to the bottom of the search results and you should hopefully see HPET_EMULATE_RTC=y.
For all the others, here’s the diff for the Debian 2.6.18 kernel:
147d146 < CONFIG_HPET_EMULATE_RTC=y 2140c2139,2141 < CONFIG_RTC=y --- > CONFIG_RTC=m > CONFIG_GEN_RTC=m > CONFIG_GEN_RTC_X=y
Please note, CONFIG_HPET_EMULATE_RTC depends on CONFIG_RTC=y.
October 27th, 2007 at 17:13
Yeah, I compiled custom kernels to try to play nice with VMWare. But then I decided that life is too short, and the kernel changes too often.
That said, it would be really excellent for you to post your kernel config (provided that it is relatively minimal apart from the RTC emulation) so that other VMWare users don’t have to go through the 50 recompiles.
Thanks very much for taking the time to comment–I amended that post: https://chxo.com/be2/20060821_3333.html
October 29th, 2007 at 20:54
Many thanks for this hints. I encounter the same problems on a Dell 860.
After trying to avoid a kernel recompilation I will go through it. Although Debian VMs running fine under the given setup I encountered the problem of running clocks in VM in rPath Virtual appliances.
A short question: Does this scenario only apply to Dell Hardware running Debian etch as host os for vmware server?
Regards
October 29th, 2007 at 22:03
Hi Ralf,
This does not only apply to Dell hardware. If anybody is experiencing these specific problems with RTC on their hardware, I think this might work.
I’ve read up a bit on HPET, and it seems Dell is one of the few companies who were the first to manufacture hardware with it.
Also, any Linux OS that displays the symptoms is affected. =)
January 6th, 2008 at 11:34
Thank you for this post. It was a major time saver and much appreciated.
January 7th, 2008 at 08:58
Apparently Debian has CONFIG_HPET_EMULATE_RTC=y set by default now (At least in 2.6.22-3-amd64 it is). Hardware is Dell PE 1435.
But I’m still getting the infamous “rtc: lost some interrupts at 512Hz.”
January 7th, 2008 at 09:00
The important part is setting “RCT” to be built-in. Have you made sure of that?
March 9th, 2008 at 06:31
WHY rebuild kernel when HPET usually can be turned off in BIOS? HPET isnt a requirement but rather a supplement, im sure HPET will be supported in the next major Debian release, until then Debian is still one of the best distros and i hope people dont start recompiling or change distro without taking 30 seconds to look for it in Bios first.
March 9th, 2008 at 15:43
th3D:
Well, in my situation (With a Dell PowerEdge server), the BIOS didn’t have an option to turn off HPET. Also, why NOT use HPET?
April 17th, 2008 at 03:11
revisiting this issue… apparently this isn’t the magic bullet…
/proc$ zcat config.gz |grep CONFIG_RTC
CONFIG_RTC=y
/proc$ zcat config.gz |grep HPET
CONFIG_HPET_EMULATE_RTC=y
/proc$ dmesg | tail -2
rtc: lost some interrupts at 512Hz.
rtc: lost some interrupts at 512Hz.
Hardware is SC1435.
Linux hostname 2.6.22-3-amd64 #1 SMP Mon Nov 12 17:53:18 UTC 2007 x86_64 GNU/Linux
Any other ideas?
April 26th, 2008 at 14:12
That is very odd. I’ve never – at all – come across this issue with 64-bit architectures.
Did you try poking in your kernel config?
July 3rd, 2008 at 15:43
I’ve been having timing issues with vmplayer on a custom kernel. I have done EVERY possible thing I could think of when building a custom kernel for vmplayer but the time always runs really fast and the reported CPU is much slower than it actually is. I read all of vmware’s documentation and all I found is that the kernel requires the rtc and parallel port support. Using the tsc as the system timer works fine for the guest (with the exception of choppy audio) but performance suffers. I would like to be able to use the hpet. Could you possibly post your entire kernel config? btw, vmware’s documentation for linux is disgusting so I appreciate posts like this.
Thanks,
Ben
September 12th, 2008 at 16:04
[...] http://nixy.dk/2007/10/24/quickpost-rtc-problems-in-debian-on-dell-poweredge-servers/ [...]