Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Server clock falls behind
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Sat Jul 27, 2013 12:45 am    Post subject: Server clock falls behind Reply with quote

What could cause my server's clock to start falling behind after setting it via ntp-client? Within hours of running ntp-client, I notice that my server's clock is a minute behind and falling further so.

My current server is a dedicated hardware server. It was built from a stage-4 tarball of when it used to be a virtual machine, and one that had known clock skew issues. But the kernel is one I built with a fresh seed when I moved to a dedicated server. I'm guessing there's a kernel setting I should adjust, but don't know which one. I'd appreciate advice.
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
vaxbrat
l33t
l33t


Joined: 05 Oct 2005
Posts: 731
Location: DC Burbs

PostPosted: Sat Jul 27, 2013 1:38 am    Post subject: hardware issue? Reply with quote

I have experiences with some HP 8xxx class workstations that were based on an early generation opteron (maybe even original slegehammer) where the mobo and an early 2.6.x kernel had bad clock skew. The only other time I've seen something like there is where line noise was coming in through the power supply and causing some sort of resonance or ringing problems with the clock crystal. That one was due to some microwave frequency interference from radars.
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Mon Jul 29, 2013 1:52 pm    Post subject: Reply with quote

I don't think I have that processor, and I have a much later kernel. Maybe I should email OVH and see if they know of any clock skew issues.
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5592

PostPosted: Mon Jul 29, 2013 5:32 pm    Post subject: Reply with quote

The kernel can account for clock drift, but to do that it needs continuous NTP updates. Running ntp-client once isn't going to work.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Mon Jul 29, 2013 5:38 pm    Post subject: Reply with quote

audiodef,

ntp-client is a one time set the clock application.
You need ntpd to check and sync the clock too.

PC crystals are very poor timekeepers and they are always slow.
Crystals the run a little fast can be trimmed down. Crystals that run slow are sold cheaply or scrapped.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Wed Aug 07, 2013 3:40 pm    Post subject: Reply with quote

Does it matter whether I use ntpd or run ntp-client as a cron job every hour?
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Wed Aug 07, 2013 7:58 pm    Post subject: Reply with quote

audiodef,

Unless your clock is so bad that ntpd cannot calibrate a correction, ntpd is preferred.
Your clock may run slow but it should be a fixed rate slow. Once ntpd has calibrated, it will keep the clock correct. ntp-client will make step changes to system time, which is in general a bad thing.

Steps forward are preferred over steps back. The latter will play havoc with make.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Wed Aug 07, 2013 9:00 pm    Post subject: Reply with quote

Interesting... I just noticed that ntpd is already started and yet I've been having clock problems. Upon restart of the daemon, I got "start-stop-daemon: no matching processes found", so maybe it got killed somehow and the pid got orphaned. It started up, and a subsequent restart had no repeat of this issue.

I've taken ntp-client out of its cron job and will see what ensues.
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Thu Aug 08, 2013 5:25 pm    Post subject: Reply with quote

audiodef,

Read man ntpd note the -g option.

If you want to look at the drift its in /var/lib/ntp/ntp.drift
I have not read enough to know if thats an offset or a rate to be applied to timekeeping.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
morpheus2051
Tux's lil' helper
Tux's lil' helper


Joined: 07 May 2006
Posts: 94

PostPosted: Fri Aug 09, 2013 8:53 am    Post subject: Reply with quote

Hi,

even if you use the -g parameter, ntpd kills itself when the clock skew gets greater than a 1000s during the time ntpd is running. I had this problem on an old kvm machine. Worked around it by calling ntpdate and then restarting ntpd every few hours. With a new kvm version this problem went away.
Back to top
View user's profile Send private message
Akkara
Administrator
Administrator


Joined: 28 Mar 2006
Posts: 6600
Location: &akkara

PostPosted: Fri Aug 09, 2013 10:56 am    Post subject: Reply with quote

If the old system had a bad clock, it's possible there's files laying around that has incorrect information for the current system. /etc/adjtime is often the culprit, if you brought the old image directly over and it had one there.

It is OK to remove that file, it'll get re-generated using new measurements of clock skew. (But, it's been a while since I had this problem and things may be different now, so check first.)
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Fri Aug 09, 2013 3:10 pm    Post subject: Reply with quote

It seems to be working OK. No major skewing in the last few days. /etc/adjtime was 0.0.0.0 but I rm'd it anyway (thanks for that advice).

It seems like -g only really helps once, but I will peruse the man page if I run into any further major skew.

I appreciate all the responses. :)
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
CleanTestr
n00b
n00b


Joined: 15 Jan 2013
Posts: 47
Location: somewhere in Rural Nebraska, USA

PostPosted: Sat Aug 10, 2013 9:33 am    Post subject: Reply with quote

Back in the 90s we used to get PCs with 'really bad clock skew' (at D+B, Inc, USA).
Using an add-on PCI-bus clock-card would solve the problem: it 'replaces'
the crystal and RTC on the mobo.

(If you don't mind spending the money, I seem to recall that, you can get
'oven-controlled crystal temperature' on some high-end add-on cards)
_________________
Stan: A signal? Why didn't you wake me?
790: It was a distress signal. They only lead to trouble, so I always ignore them. --Lexx
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Sat Aug 10, 2013 11:56 am    Post subject: Reply with quote

CleanTestr,

If you don't mind how much you spend, you can get an atomic clock for a PC.
Of course, it doesn't fit inside the PC case :)
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Sat Aug 10, 2013 2:10 pm    Post subject: Reply with quote

Get a bigger case? :P
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Sat Aug 10, 2013 3:34 pm    Post subject: Reply with quote

Don't use cron for this. And take this opportunity to switch from ntpd to chrony, as Fedora and Mageia have done.
Back to top
View user's profile Send private message
CleanTestr
n00b
n00b


Joined: 15 Jan 2013
Posts: 47
Location: somewhere in Rural Nebraska, USA

PostPosted: Sat Aug 10, 2013 5:32 pm    Post subject: Reply with quote

@ NeddySeagoon:

that would be the 'older style' Cesium Atomic Clock, would it not?
_________________
Stan: A signal? Why didn't you wake me?
790: It was a distress signal. They only lead to trouble, so I always ignore them. --Lexx
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Sat Aug 10, 2013 5:35 pm    Post subject: Reply with quote

CleanTestr,

Yep.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6308
Location: /usr/lib64/lv2

PostPosted: Sun Aug 11, 2013 2:58 pm    Post subject: Reply with quote

PaulBredbury wrote:
Don't use cron for this. And take this opportunity to switch from ntpd to chrony, as Fedora and Mageia have done.


I've take out the ntp-client cron job.

Thanks for the tip on chrony. I read the linked page, and none of the advantages seem to apply to my server: it's not a laptop, it's always on, it has loads of bandwidth and memory, and it isn't a VM. Given that, would chrony still have any real advantage over ntpd?
_________________
Gentoo Studio: A Gentoo-based, professional digital audio workstation OS.
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Sun Aug 11, 2013 3:06 pm    Post subject: Reply with quote

audiodef wrote:
any real advantage over ntpd?

I don't know of any disadvantages, so what's not to like?

Less RAM usage, no clock stepping - these are advantages on any PC, you're not somehow special by having a "server" ;)
Back to top
View user's profile Send private message
CleanTestr
n00b
n00b


Joined: 15 Jan 2013
Posts: 47
Location: somewhere in Rural Nebraska, USA

PostPosted: Sun Aug 11, 2013 4:14 pm    Post subject: Reply with quote

@ audiodef:

Not to rain on ne1's parade, but, though finding a purely software solution to this
problem might be nice, in Theory, perhaps some consideration ought be given to
the underlying problem: that, once set via ntp, there is clock drift.

Crystals are simple beasts: they're affected by current, and by temperature.
On a larger server, it's possible that the cooling given to the cpu is not also applied
to the part of the mobo housing the crystal.

(not knowing your configuration) In the past I've run AT-style cases with 2 additional
case fans (one front, one rear) in addition to the power supply fan; and in rack-
mount equipment (such as raid), as many as one fan per 3 drives in a cage.

An example that might make sense for large cooling heatsinks on cpu chips is:
when there is side-flow from the heat-sink fan (out horizontally across the surface
of the mobo, for instance). If the crystal is in the path of such outflow, it could be
running at 2x or higher its rated operating temperature.

You might want to take a look at better air-flow within the server, perhaps.
_________________
Stan: A signal? Why didn't you wake me?
790: It was a distress signal. They only lead to trouble, so I always ignore them. --Lexx
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Sun Aug 11, 2013 5:57 pm    Post subject: Reply with quote

CleanTestr,

Crystals are manufactured to some random tolerances but the frequency distribution will be gaussian for any reasonable process.

Those that run too fast, can be trimmed down but the trimming has an adverse effect on lots of other parameters, like stability and temperature coeffcient of frequency. These crystals do not find their way into PCs.
Crystals that run too slow are either scrap or sold for use in applications where frequency precision isn't too important, like PC timing circuits.
Here, we are talking about the CPU frequency control, not the real time clock, which is usually pretty good.

After boot up and the kernel has synced its real time to the RTC, the kernel keeps its real time updated from the rather poor low frequency crystal used for the the CPU clock.
No amount of cooling will fix it - its already too slow.

Hence the software calibration and correction approach.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Jaglover
Watchman
Watchman


Joined: 29 May 2005
Posts: 6957
Location: Saint Amant, Acadiana

PostPosted: Sun Aug 11, 2013 6:05 pm    Post subject: Reply with quote

I have a M/B that just couldn't keep the clock right with openntpd. It kept fluctuating too fast / too slow. It is a desktop M/B with mobile CPU. I switched to net-misc/ntp and haven't had a clock issue after that. My 2¢.
_________________
Please learn how to denote units correctly!
Back to top
View user's profile Send private message
CleanTestr
n00b
n00b


Joined: 15 Jan 2013
Posts: 47
Location: somewhere in Rural Nebraska, USA

PostPosted: Sun Aug 11, 2013 9:11 pm    Post subject: Reply with quote

@ NeddySeagoon:

Well, * if it were Me *, I'd be in there with a temperature probe and
an oscilloscope... :P

Not saying that that's the way to go, in this case... ~~P. -- * ~.

Ohhh, well.
_________________
Stan: A signal? Why didn't you wake me?
790: It was a distress signal. They only lead to trouble, so I always ignore them. --Lexx
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 42564
Location: 56N 3W

PostPosted: Sun Aug 11, 2013 9:27 pm    Post subject: Reply with quote

CleanTestr,

Been there done that ... but not on a PC.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum