RSS

Tag Archives: linux

Commonly used daily Linux Commands.

1) How to Replace a big string with WildCard Characters.  just dump the string in between the hash (#) and  it does the job.  use /g at the end to replace strings everywhere.

%s#/this/folder/replaces/#/with/this/one/#g

2) Search for a word and delete to the end of the line.  for ex: in an XML file  sometimes you need to delete 1000’s of lines matching a word or a <string> spread across the XML file,  you can’t simply replace some lines like below ex.. in MS-WORD or any other editors,   from the below ex: only delete those which starts with <pcode> until the end of the line and preseve the “\>”. use the below simple command to do the job.

<xml=? >
<scalar variable   <pcode  value=1000  test1 test3 test4> />
<vector variable   <pcode  value=1001  test5 test6 test7> />
<stellar value       <pcode  value=1002  test8 test9 test10> />
</xml>

%s/\<pcode\>.*/\/>/

3)  How to find Un-wanted files, if you’r clean folder has been messed up.  I had this situation where some of the junk files has been added to my MySql Data direcotory where i see only the Database Tables which are useful and i used the below command to find all the files excluding MySql Database Tables.

grep ./ --exclude=*.{ibd,MYD,MYI,frm} *

4)  CHOP the file to 1gb from a 9.5GB file : i had this big file which is 9.5gb Data MySql Log file for my analysis purposes, however my script takes too long time to read this file and i had no choice to chop this file to 1GB and read the data from this 1GB file which makes it easier for MySql to read faster.  I used the below command.

 dd if=10gbfilename of=1gb_new_filename bs=100M count=10

5)  How to get extract 100 lines of data from a file which has 10,000 lines.
sed -n 1,100p test1.log > outputfile.log
6) How to find which raid your Linux software has..

for i in /dev/md*; do printf ‘%s: %s\n’ $i “$( sudo /sbin/mdadm –detail $i 2>/dev/null | grep ‘Raid Level’ )”; done

7) Convert files to Unix &  UTF8 format.

Convert to UTF8 format

/usr/bin/iconv -c -f LATIN1 -t UTF8 insert_statements_postgres1.sql > utf8_postgres_inserts.sql

/usr/bin/iconv -c -f LATIN1 -t UTF8 delete_statements_post1.sql > utf8_postgres_deletes.sql

Convert Bulk files to utf8 format: (csv files)

for file in *.csv; do
/usr/bin/iconv -c -f LATIN1 -t UTF8 “$file” -o “${file%.csv}.csv”
done

 
 

Tags: , , , , , ,

Subversion Setup

http://www.eukhost.com/forums/f15/how-install-subversion-svn-linux-server-vps-3737/

http://marcgrabanski.com/article/installing-subversion-on-apache

  1. yum install subversion
  2. yum install mod_dav_svn

Creating a User in Svn using the bash Command

1) htpasswd -cm users sam
2) htpasswd -cm users chava
3) htpasswd -cm users veera

Importing a Folder to Repository in Svn (to checkout for later use)

1) svn import -m "Initial Import" local-directory-projectX http://server-name/svn/projectX

2)

follow this link to install subversion Server 1.4 on Linux Server with Apache WebServer Running

http://www.yolinux.com/TUTORIALS/LinuxSubversionAndTracServer.html

Create a Subversion repository using the FSFS database format: (as root)

  • mkdir /srv/svn/repos
  • svnadmin create --fs-type fsfs /srv/svn/repos
  • chown apache:apache -R /srv/svn/repos
    (The Apache web server running as user “apache” is the process owner which will be interfacing with the Subversion repository.)
  • SELinux security attribute assignment: (Red Hat EL4+, Fedora 3+)
    restorecon -R /srv/svn/repos

Apache configuration file: /etc/httpd/conf.d/subversion.conf

  • This example performs no authentication:
    LoadModule dav_svn_module     modules/mod_dav_svn.so       - Required for Apache interface
    LoadModule authz_svn_module   modules/mod_authz_svn.so     - Required for "per-directory" access control
    <Location /svn>        - URL path in web browser after "http://domain.com/"
       DAV svn
       SVNPath /srv/svn/repos        - Local path on server filesystem
    </Location>
  • This example authenticates to a local Apache user password file: User logins and passwords have no connection to user accounts.
    LoadModule dav_svn_module     modules/mod_dav_svn.so
    LoadModule authz_svn_module   modules/mod_authz_svn.so
    <Location /svn>
       DAV svn
       SVNPath /srv/svn/repos
       AuthType Basic
       AuthName "Subversion Repository"
       AuthUserFile /etc/httpd/conf/userpw
       require valid-user
    </Location>

    Remove the line “require valid-user” to allow read-only access for unauthenticated users.

Importing Directory of Files Into Subversion:

Subversion Repository on Server File system:

  • /srv/svn/repos

Import directory of files into Subversion:

  • Local directory to import into Subversion:
    • projectX/trunk/...source code goes here...
    • projectX/branches
    • projectX/tags
  • svn import -m "Initial Import" local-directory-projectX http://server-name/svn/projectX
    The import will create the new subdirectory branch “projectX” in the repository.

Alternate directory creation method: (repository exists but the directories need to be generated)

Browser view of a typical repository directory schema:

  • http://svn-server/svn/projectX/trunk: For the “HEAD”
  • http://svn-server/svn/projectX/branches: For branches or “forks” in the code.
  • http://svn-server/svn/projectX/tags: For tags identifying snapshots of milestones or significant releases.

Explanation of Subversion directories:

  • /trunk : Contains the current development branch. New projects should be started here.
  • /branches : A branch is just a special copy of the source tree within Subversion that started with a specific revision of the code. A branch is created by using “svn copy” of the trunk to a branch i.e. /branch/name_of_branch_1.0.
  • /branches/personal : Personal branches are for doing some work which you don’t want to have interfere with the main trunk until it is working or better defined.
  • /tags : This is a ethod of bookmarking or taking a “snapshot” of some work in progress.
  • /releases : This is similar to “tags” except that this version of the code makes it to production. Names used here should match product release names. i.e. version_1.1


Avoiding false diffs due to “^M” in a cross platform environment:

The Microsoft development tools love to add a “^M” at the end of the line of every file they edit. This breaks UNIX shell script and causes many file difference tools to show a difference in a line of code. This can be avoided in a cross platform environment by telling Subversion that certain files (or all files) that no “^M”‘s should be appended at the end of a line. The trigger in fact removes the “^M” when the file is checked in. It can also be removed using the command dos2unix.

Import files to support no “^M” carriage returns at the end of lines:
(This sets file properties and creates a check-in trigger.)

  • Remove “^M” from files: find local-directory -name "*.cpp" -exec dos2unix {} \;
  • Upload directory of files into Subversion:
    svn import -m "Initial Import" local-directory http://server-name/svn/projectX
  • Checkout files from repository: svn co http://server-name/svn/projectX/trunk
    This creates your local working directory under Subversion control.
  • Set file properties such that “^M” are removed during check-in if added:
    find ./ -name "*.cpp" -exec svn propset svn:eol-style LF {} \;
  • Apply property changes to repository: svn ci -m "Apply LF properties" local-directory

Note that the “propset” command must be used for new files added to the Subversion repository if they are to have these properties.

Users can also set this option in the file: $HOME/.subversion/config

..
...

[auto-props]
README = svn:eol-style=native
INSTALL = svn:eol-style=native
*.c = svn:eol-style=LF
*.cpp = svn:eol-style=LF
*.h = svn:eol-style=LF
*.dsp = svn:eol-style=CRLF
*.dsw = svn:eol-style=CRLF
*.sh = svn:eol-style=native;svn:executable
*.txt = svn:eol-style=LF
*.png = svn:mime-type=image/png
*.jpg = svn:mime-type=image/jpeg
Makefile = svn:eol-style=LF
*.html = svn:eol-style=LF
*.css = svn:eol-style=LF
*.java = svn:eol-style=LF
*.xml = svn:eol-style=LF
*.m4 = svn:eol-style=LF
*.pdf = svn:mime-type=application/pdf

...
..

Data repository dump:

  • Dump first entry to current: svnadmin dump /srv/svn/repos --revision 0:HEAD > repos.dump
  • Dump revision 625 to current: svnadmin dump /srv/svn/repos --revision 625:HEAD --incremental > repos-625.dump
  • *************************************************************************************************************
TROUBLESHOOTING SUBVERSION. --------------------------- use the below path run the below commands first to make sure you are giving access to the repos folder. use these vi commands to edit the file. vi filename. examples: http://www.cs.colostate.edu/helpdocs/vi.html chown apache:apache -R /srv/svn/repos restorecon -R /srv/svn/repos edit this file and add the below script. /etc/httpd/conf.d/svnserve.conf enable these statements below anon-access = read auth-access = write <Location /svn> DAV svn SVNPath /srv/svn/repos AuthType Basic AuthName "Subversion Repository" AuthUserFile /srv/svn/access/users require valid-user </Location>


 
Leave a comment

Posted by on July 29, 2009 in Database Administration

 

Tags: , , ,

Shell Script Commands

Search for something in Linux

rpm -q -a | grep mysql   (searches for mysql files)
rpm -q -l mysql          (lists out the path of mysql search string) 

Load Putty first and Connect to the Remote Host using Remote Username and Password

1) pwd : to locate the absolute path
2) Ls -l   : list all the directory info.
3) Ls : list all the directory info in wide format.
4)  ./  : used to execute shell script files with “.sh” extension.
5) cp : copy directory from one path to another
6) cd .. : change directory just like any other Dos Directory.

htpasswd -m /etc/.svn-auth-file <username> — to create user account(must be root)  is to create a username and password from apache to use with Subversion.

http://marcgrabanski.com/article/installing-subversion-on-apache (subversion installation instructions).

actual path is    “htpasswd -m /srv/svn/access/users  username  (pressing enter will ask for password followed by password confirmation and will be saved into the access file.

# service httpd restart
(will restart the apache server).

 
2 Comments

Posted by on July 27, 2009 in Database Administration, shell script

 

Tags: , ,

 
%d bloggers like this: