RSS

Extracting 1 week of Data from a Big log File

23 Aug

It is very difficult to read one week of data from a Big Log file (i.e 2gb size log file).  where it takes 100% cpu and blocks all the processess, and take s more time to get a small piece of data using less command in unix.   to resolve this, search for the first line number of the 1st day of the Week and last line no of the last day of the Week and use SED command to extract from starting line position to end line position, copy that data to a new file.

First Try to Find the Line No’s of the Starting Date of 1 week (lets say 9 Apr 2010 – 16 Apr 2010)

grep -n ‘Time: 100409’ /var/log/mysqld/myserver_slow_queries.log | more

Copy or write down the Line no. let’s say 2399098

grep -n ‘Time: 100416’ /var/log/mysqld/myserver_slow_queries.log | more

Copy or write down the Last Line no. 2483712

now run the SED Unix Command to grab the data from start line to end line into a new log file.

sed -n 2399098,2483712p myserver_slow_queries.log > myserver_slow_queries.log.week

Done. One Week of Data from a Big Log file will be copied to a new file.

Advertisements
 
Leave a comment

Posted by on August 23, 2010 in Database Administration, linux, MySql

 

Tags: , , , ,

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
%d bloggers like this: