Download website with wget in linux

wget is a nice tool for downloading resources from the internet. The basic usage is wget url:

wget http://linuxreviews.org/

Therefore, wget (manual page) + less (manual page) is all you need to surf the internet. The power of wget is that you may download sites recursive, meaning you also get all pages (and images and other data) linked on the front page:

wget -r http://linuxreviews.org/

But many sites do not want you to download their entire site. To prevent this, they check how browsers identify. Many sites refuses you to connect or sends a blank page if they detect you are not using a web-browser. You might get a message like:

Sorry, but the download manager you are using to view this site is not supported. We do not support use of such download managers as flashget, go!zilla, or getright

Wget has a very handy -U option for sites like this. Use -U My-browser to tell the site you are using some commonly accepted browser:

wget -r -p -U Mozilla http://www.stupidsite.com/restricedplace.html

The most important command line options are --limit-rate= and --wait=. You should add --wait=20 to pause 20 seconds between retrievals, this makes sure you are not manually added to a blacklist. --limit-rate defaults to bytes, add K to set KB/s. Example:

wget --wait=20 --limit-rate=20K -r -p -U Mozilla http://www.stupidsite.com/restricedplace.html

A web-site owner will probably get upset if you attempt to download his entire site using a simple wget http://foo.bar command. However, the web-site owner will not even notice you if you limit the download transfer rate and pause between fetching files.

Use --no-parent


--no-parent is a very handy option that guarantees wget will not download anything from the folders beneath the folder you want to acquire. Use this to make sure wget does not fetch more than it needs to if just just want to download the files in a folder.


Source : http://linuxreviews.org/quicktips/wget/

Block anything based on url keywords by squid proxy

Hai guys/gals,

Here i going to tell about block any sites based on keywords squid proxy.

This help for system administrators in schools and colleges

Here i use linux sever( RHEL5 ) and squid 2.6 stable version.

It simple to do,

Step 1 : Just store what all are key words you want to block

Here i stored keywords in "/etc/squid/badkeys"

Step 2 : Using vim , open squid configuration file

Ex. vim /etc/squid/squid.conf

Step 3 : Use url_regex to block, (Just add following line in acl )

acl badkey url_regex "/etc/squid/badkey"

Ex:

#Recommended minimum configuration:
acl all src 0.0.0.0/0.0.0.0
acl your_network_name src 10.1.1.0/24
acl badkey url_regex "/etc/squid/badkey"

Step 4 : Add following line for deny badkey in http_access

http_access deny badkey

Ex:

http_access allow localhost
http_access deny badkey
http_access allow your_network_name
http_access deny all

Step 5 :That's all. Restart Squid

Hide Blogger Navbar

Step 1: Log in to Blogger

Step 2 : On your Dashboard, click Layout under Manage Blogs. Then click Edit HTML.

Step 3 : You will get HTML of your blog template.

Step 4 : Paste following code in your html head part ( paste the CSS in the top of the Variable definitions )

#navbar-iframe {
display: none !important;
}

Step 5 : Click Save

Clear cache in Linux Server

If you running Proxy Server, Mostly it'll store frequently visiting sits content.

It's one of the best feature for fast browsing, But some few cases it not good.

Here a commend for clear cache files from server,

But it not useful at all conditions,

Step 1 : Open terminal and type following

root@localhost~] # echo 3 > /proc/sys/vm/drop_caches

Step 2 : Hit Enter


That's all.


Note : It's not give solution for all problem related with it.

Install and Enable SNMP Service in Windows XP, Vista and 2003

SNMP (Simple Network Management Protocol) is an internet protocol used in network management systems to monitor network-attached devices such as computers, servers,routers, switches, gateways, wireless access points,VoIP phones, and etc. for conditions that warrant administrative attention. SNMP provides management data in the form of variables on the managed systems, which describe the system configuration parameter or current status value. These variables can then be read and queried (or sometimes set or write) by managing applications. Windows system running XP, Vista or 2003 does not turn on SNMP service by default, thus users need to manually install and enable SNMP service.

Note that you must be logged on as an administrator or a member of the Administrators group in order to complete this procedure. If your computer is connected to a network, network policy settings may also prevent you from completing this procedure.

How to Install and Enable the SNMP Service

1. In Windows XP and Windows 2003, click Start button, then go to Control Panel and run Add or Remove Programs applet. On Add or Remove Programs dialog, click Add/Remove Windows Components to open Windows Components wizard.In Windows Vista, click Start button, then go to Control Panel. Click on Programs link and then click on Turn Windows features on or off. If you’re prompted with User Account Control dialog, click “Continue”.
2. In Components of Windows XP and 2003, click on the Management and Monitoring Tools (make sure that you do not select or clear, tick or untick its check box to change the existing selection), and then click Details.In Windows Features of Vista, locate SNMP feature.
3. Select and tick the check box of Simple Network Management ProtocolSNMP feature. or
4. Click OK. Also click Next if you’re in Windows XP or 2003. SNMP service will be installed on the system. You may require to insert the Windows setup CD/DVD disc into optical drive.
5. SNMP will start automatically after installation. But it’s recommended to verify the service status from Services in Control Panel, and if it’s stopped, you can start the SNMP service from there.

Two new services will be created:

1. SNMP Service which is the main engine with agents that monitor the activity in the network devices and report the information to the monitoring console workstation.
2. SNMP Trap Service which receives trap messages generated by local or remote SNMP agents and forwards the messages to SNMP management programs running on this computer.

Windows doesn’t assign any community string to the SNMP service by default, and also only allow access from localhost or local devices. Further configuration is needed to add in desired community string, which act as the password to grant reply to any SNMP request from remote system.

How to Configure SNMP Service (Add “public” community string)

1. Click on Start button, then go to Control Panel.
2. In Windows Vista, click on System and Maintenance link.
3. Open Adminstrative Tools.
4. Run Services applet.
5. Locate and right click on SNMP Service, then select Properties.
6. In SNMP Service Properties window, click on Traps tab.
7. In the “Community name” text box, enter public or any other case-sensitive SNMP community name to which this computer will send trap messages.
8. Click on Add to list button. 

How to Configure Security for SNMP Service for a Community

1. Continue from above steps, click on Security tab. If you already close SNMP Service Properties window, re-open it.
2. Under “Accepted community names” section, click Add button.
3. Select the appropriate permission level for the community string in the “Community Rights” drop down list to specify how the host processes SNMP requests from the selected community. Normally READ ONLY is recommended.
4. In the “Community Name” box, type public or any case-sensitive community name that you want.
5. Click on Add button.
6. In order for the SNMP service to accept and receive SNMP request packets from any host on the network, including external remote host regardless of identity, click Accept SNMP packets from any host.To limit the acceptance of SNMP packets, click Accept SNMP packets from these hosts, and then click Add, and then type the appropriate host name, IP or IPX address in the Host name, IP or IPX address box. You can restrict the access to local host or limited servers only by using this setting. Finish off by clicking Add button again.
7. Click OK when done. Note that you may need to reboot for the settings to take effect.

Installing Open Office org in RHEL5

First want to download Open office ,

And save it in desktop ( for basic user )

Use following comment to extract package ,

tar zxvf OOo_{VERSION}_LinuxIntel_install_en-US.tar.gz

---- eg. #tar zxvf OOo_3.1.1_LinuxIntel_install_wJRE_en-US.tar.gz

Now want to install rpm, for that first goto rpm folder

cd SOMEDIR/RPMS

---- eg. #cd OOO310_m19_native_packed-1_en-US.9420/RPMS/

After that you install the rpms by doing

#su -

#rpm -ivh *.rpm desktop-integration/*redhat*

have to replace the {VERSION} part with the actual version listed in the file you downloaded. Same thing with SOMEDIR, replace that with the name of the actual directory that gets extracted after running the first command.

Creating DVD repository for own system or server

1st Mount RHEL5 dvd

1)check :

#rpm -qa | grep vsftpd

-if /var/ftp directory didn't there means install

#rpm -ivh /media/RHEL5/Server/vsftpd* ( this is location of file in mounted dvd )


2) check :

#rpm -qa | grep createrepo

-if createrepo command not installed means install

#rpm -ivh /media/RHEL5/Server/createrepo* ( this is location of file in mounted dvd )


3)Copy dvd/cd files to pub:

#cp -avf /media/RHEL/* /var/ftp/pub/


4)copy grouping files:

#cp /var/ftp/pub/Server/repodata/comps-rhel5-server-core.xml /root

#cp /var/ftp/pub/VT/repodata/comps-rhel5-vt.xml /root


5)clear the old repos:

#rm -rf /var/ftp/pub/Server/repodata

#rm -rf /var/ftp/pub/VT/repodata


6)repo creation:

#createrepo -g /root/comps-rhel5-server-core.xml -v /var/ftp/pub/Server

#createrepo -g /root/comps-rhel5-vt.xml -v /var/ftp/pub/VT


7) create a file called :

for server:

#vim /etc/yum.repos.d/server1.repo

[Server]

name=Server1 Server Repository

baseurl=file:///var/ftp/pub/Server

gpgcheck=0

#This one is needed for xen packages

[VT]

name=Server1 VT Repository

baseurl=file:///var/ftp/pub/VT

gpgcheck=0


for client:

#vim /etc/yum.repos.d/client.repo

[Server]

name=Server1 Server Repository

baseurl=ftp://192.168.0.254/pub/Server

gpgcheck=0

# This one is needed for xen packages

[VT]

name=Server1 VT Repository

baseurl=ftp://192.168.0.254/pub/VT

gpgcheck=0


8) checking

#yum clean all

#yum install