Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Analyze log - show last hour
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  
Author Message
bastibasti
Guru
Guru


Joined: 27 Nov 2006
Posts: 498

PostPosted: Mon Oct 21, 2013 9:07 am    Post subject: Analyze log - show last hour Reply with quote

Hi,

is the a quick way of displaying the last hour of logfile ?? I could write a long bash script for that but maybe there is already a way of doing it.
Reason: I want to see, which clients renewed their dhcp leases within the last hour to see if they are still there.
I have a couple of clients that nmap will not find as they go to sleep (iPhones) but I need to know whether they are present

Any other ideas are welcome
Back to top
View user's profile Send private message
notageek
Tux's lil' helper
Tux's lil' helper


Joined: 05 Jun 2008
Posts: 120
Location: Bangalore, India

PostPosted: Mon Oct 21, 2013 9:26 am    Post subject: Reply with quote

Depending on which log you're trying to extract and if you're using Linux you can try this.

Code:
grep "$(date +"%b %d %H:" -d "1 hour ago")" /var/log/syslog


Basically greps for: "Oct 21 13:"

Example log extract:

Code:
Oct 21 13:17:01 localhost CRON[14665]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)

_________________
The problem is not the problem. The problem is your attitude about the problem. Do you understand? --Capt Jack Sparrow.
Back to top
View user's profile Send private message
bastibasti
Guru
Guru


Joined: 27 Nov 2006
Posts: 498

PostPosted: Mon Oct 21, 2013 9:51 am    Post subject: Reply with quote

Thanks, yes syslog on linux

your method is understood ,but it is not exactly what I need...

If now is "Oct 21 11:48" that would give me everything from 10:00 to 10:59 - but I need everything from 10:48 - 11:48
OF course I could do it with a for loop etc but it would consume quite some time and I wonder if there's an easy way to do this.
Back to top
View user's profile Send private message
notageek
Tux's lil' helper
Tux's lil' helper


Joined: 05 Jun 2008
Posts: 120
Location: Bangalore, India

PostPosted: Mon Oct 21, 2013 10:02 am    Post subject: Reply with quote

Code:
min=$(date +%M|sed s'/.$//'); min2=$(date +%M|sed s'/^.//'); egrep "$(date +"%b %d %H:" -d "1 hour ago")[$min-5][$min2-9]|$(date +"%b %d %H:")" /var/log/syslog

_________________
The problem is not the problem. The problem is your attitude about the problem. Do you understand? --Capt Jack Sparrow.
Back to top
View user's profile Send private message
bastibasti
Guru
Guru


Joined: 27 Nov 2006
Posts: 498

PostPosted: Mon Oct 21, 2013 11:30 am    Post subject: Reply with quote

Oh WOW!

I need to checkout egrep more detailed. THANKS!
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16114
Location: Colorado

PostPosted: Mon Oct 21, 2013 3:24 pm    Post subject: Reply with quote

I like the grep family of utilities a lot. Sometimes it becomes too convoluted.

Code:
awk '/Oct 21 10:48/,/Oct 21 11:48/' messages



notageek, I'm surprised you didn't use perl! :P
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Prenj
n00b
n00b


Joined: 20 Nov 2011
Posts: 13

PostPosted: Mon Oct 21, 2013 9:47 pm    Post subject: Reply with quote

People still parse logs? Get Splunk instead
_________________
“If You Meet the Buddha on the Road, Kill Him”
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16114
Location: Colorado

PostPosted: Tue Oct 22, 2013 1:21 am    Post subject: Reply with quote

Quote:
Gartner names Splunk a leader in the 2013 Magic Quadrant for Security Information and Event Management.
lol

Sounds designed for selling to PHBs.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1566
Location: U.S.A.

PostPosted: Tue Oct 22, 2013 2:32 am    Post subject: Reply with quote

Pointy-haired bosses know they can spend less on such a tool than on the risk of you cobbling together some shit in perl that only you understand and which they can't get to run any more two months after they fire you.
_________________
pjp wrote:
I didn't misquote you, I just misunderstood you.
Back to top
View user's profile Send private message
notageek
Tux's lil' helper
Tux's lil' helper


Joined: 05 Jun 2008
Posts: 120
Location: Bangalore, India

PostPosted: Tue Oct 22, 2013 2:33 am    Post subject: Reply with quote

:lol: :lol:
_________________
The problem is not the problem. The problem is your attitude about the problem. Do you understand? --Capt Jack Sparrow.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1566
Location: U.S.A.

PostPosted: Tue Oct 22, 2013 2:55 am    Post subject: Reply with quote

Hey, sometimes the truth hurts. It's called "capturing institutional knowledge" and "automation tactics which facilitate persistent organizational learning". :P
_________________
pjp wrote:
I didn't misquote you, I just misunderstood you.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16114
Location: Colorado

PostPosted: Tue Oct 22, 2013 3:41 am    Post subject: Reply with quote

I'm sure there are other tools out there without the PHB soft-porn. They're likely to actually work, too.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
Prenj
n00b
n00b


Joined: 20 Nov 2011
Posts: 13

PostPosted: Tue Oct 22, 2013 6:28 am    Post subject: Reply with quote

pjp wrote:
I'm sure there are other tools out there without the PHB soft-porn. They're likely to actually work, too.

Clever, but splunk actually works. Want to extract events from a certain category within a certain time frame, and filter out stuff you don't need? You just like do it. Fuck your archaic awk skills son, thats like revving between gear shifts. Evolution.
:lol:
_________________
“If You Meet the Buddha on the Road, Kill Him”
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1566
Location: U.S.A.

PostPosted: Tue Oct 22, 2013 8:20 am    Post subject: Reply with quote

Sometimes the right tool for the job is a helicopter, sometimes it's a paper-clip.

The nice thing about being able to use command-line tools and being able to code is that you can investigate any kind of one-off issue and create your own tools, just right for any job (at the cost only of your time).

The nice thing about packaged software is that, often, people have the similar problems and can use similar solutions, and there's a big economy of scale to be had in re-using what a lot of people have contributed to and in not reinventing the wheel.

I once saw a consulting firm send an analyst to Microsoft Access training because one of the Partners had it in his head that he needed a "database" to run some "queries" against. It took the poor guy three weeks to build the database, and it turned out to be a one-time requirement (and a spreadsheet would have done just fine).

This also reminds me of one of my clients (a large steel conglomerate) who had their own custom-built office suite, written from scratch, and a team of eleven people maintaining and enhancing it (and it looked like something from 1993).

Also reminds me of one time when cokehabit came in here complaining about Firefox running slowly. Then, for some reason we saw a screenshot, and he had like 18 add-ins running (I do not exaggerate). :P
_________________
pjp wrote:
I didn't misquote you, I just misunderstood you.
Back to top
View user's profile Send private message
Prenj
n00b
n00b


Joined: 20 Nov 2011
Posts: 13

PostPosted: Tue Oct 22, 2013 1:10 pm    Post subject: Reply with quote

I agree with you and pjp, just saying that in an environment where you have bazillion nodes, servers, whatnot, it's cool to be able to access events from the whole environment at one place using queries and expressions instead of sshing into some machine grepping files and all that pedestrian stuff.

Especially if you are NOT an admin, it's not only admins that need access to live information, and some admins are so dense that I'd rather bypass them and extract information I need myself. All that one needs is granted access on a class of information, which is less of a security issue (and procedure) than getting your public key onto live machines.

It's a difference between geting the information in 10 minutes, or at admins convenience (next day, if lucky).

For example, at a payment solutions company, I can extract live data (purchases and whatnot) in a prefered format (json), anonymize it, and replay the traffic pattern in test environment as a part of performance tests. So the option is to log into Splunk, enter the expression that defines what events I'm interested in, and get the data. Or I could send an e-mail to an admin and get a reply in style of "What?"
_________________
“If You Meet the Buddha on the Road, Kill Him”
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16114
Location: Colorado

PostPosted: Tue Oct 22, 2013 4:20 pm    Post subject: Reply with quote

Prenj wrote:
You just like do it.
lol, just like awk.

If you say it works, I'll take your word for it. But if I'm going to pay for something, I'm not interested in 90% of that being marketing voodoo.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
bastibasti
Guru
Guru


Joined: 27 Nov 2006
Posts: 498

PostPosted: Tue Oct 22, 2013 4:57 pm    Post subject: Reply with quote

Quote:
People still parse logs?


yeah I do, for example to check who is in the house. if no mobile phones there for one hour you can turn the light off ;-))
Back to top
View user's profile Send private message
marens
Apprentice
Apprentice


Joined: 05 Aug 2004
Posts: 172

PostPosted: Tue Oct 22, 2013 9:31 pm    Post subject: Reply with quote

Prenj wrote:
People still parse logs? Get Splunk instead


People still pay for splunk? http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd
_________________
If English was good enough for Jesus, then it's good enough for you!
Back to top
View user's profile Send private message
Prenj
n00b
n00b


Joined: 20 Nov 2011
Posts: 13

PostPosted: Tue Oct 22, 2013 10:43 pm    Post subject: Reply with quote

marens wrote:
Prenj wrote:
People still parse logs? Get Splunk instead


People still pay for splunk? http://docs.fluentd.org/articles/free-alternative-to-splunk-by-fluentd

Cool!
_________________
“If You Meet the Buddha on the Road, Kill Him”
Back to top
View user's profile Send private message
wildhorse
Tux's lil' helper
Tux's lil' helper


Joined: 16 Mar 2006
Posts: 148
Location: Estados Unidos De América

PostPosted: Wed Oct 23, 2013 1:35 pm    Post subject: Reply with quote

Quote:
Prerequisites
Java for ElasticSearch
Instant gastrospasm.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum