Linux Recursive GREP search excluding directories example

This command will search for text inside files in a directory while excluding some directories you do not want to search in.

This is helpful when a search is going slow due to a directory that might contain thousands of files that you are not interested in searching through, such as an images or cache directory.

grep -r –exclude-dir=product_images –exclude-dir=cache –exclude-dir=.svn search term *

 

 

 

How to use whatis and xargs linux commands

Have you ever wanted to know what the commands various commands in linux do?  Such as the commands in:

/bin
/sbin
/usr/bin

Move to the bin directory of your choice and follow the example below:

cd /bin
ls | xargs whatis | less

You will get an output of the command and a summary of it’s function:

 

basename             (3p)  – return the last component of a pathname

bash                 (1)  – GNU Bourne-Again SHell

bash [builtins]      (1)  – bash built-in commands, see bash(1)

bash [sh]            (1)  – GNU Bourne-Again SHell

ca                   (1ssl)  – sample minimal CA application

cat                  (1)  – concatenate files and print on the standard output

cat                  (1p)  – concatenate and print files

chgrp                (1)  – change group ownership

chgrp                (1p)  – change the file group ownership

chmod                (1)  – change file mode bits

chmod                (1p)  – change the file modes

chmod                (2)  – change permissions of a file

chmod                (3p)  – change mode of a file

chown                (1)  – change file owner and group

 

The netcat linux command

One of the neatest tools that I’ve found is the program netcat. It is a way to pipe data over the network simply and easily. You don’t need a fancy protocol like CIFS, just stream the data over the LAN.

Netcat is a “full duplex” connection, meaning it both sends and receives using the connection. It “cross-connects” the standard in of one computer to the standand out of the other and vice versa.

You specify one computer as the listener, and one as the receiver.

Type “echo hi there | nc -v -l -p 5000″ on one computer, then “echo receiving you loud and clear | nc -v 192.168.1.1 5000″ on the other.

On the “listening side”:

nc -v means verbose (tells you what ports are being listened on)
-l means listen
-p specifies the port to listen on
5000 specifies the port number

nc -v means verbose (tells you what ports are being connected to)
192.168.1.1 is the IP address to connect to
5000 is the IP port number to connect.

If you want to use the same computer, open up two lxterminal windows and pretend they are different computers, using “localhost” or “127.0.0.1” as your IP address.

Type “echo hi there | nc -v -l -p 5000″ in one terminal window, then “echo receiving you loud and clear | nc -v 127.0.0.1 5000″ on the other.

It really doesn’t matter who is the “listener” because as I mentioned, netcat “cross connects” the standard input of one side to the standard output of the other side.

There are examples abound on the internet, but here’s one that I particularly like: imaging hard drives over the network.

So on one computer (the one to image) I type: (note that the -q 0 is necessary to close the link when stdin is closed, otherwise the
connection will stay open)

“dd if=/dev/hda | nc -v -l -p 5000 -q 0″

and on the destination computer I type:

“nc 192.168.1.1 5000 > /media/sda1/MY_IMAGE_FILE”

And if you want to reverse it, where the sending computer is the “client” if you will,

type this on the destination computer first (because it has to be listening before you connect to it)
“nc -v -l -p 5000 > /media/sda1/MY_IMAGE_FILE”

and

“dd if=/dev/hda | nc 192.168.1.2 5000 -q 0″ on the sending computer.

If you have a knoppix with the pv program (pipe viewer) you can throw a pv into the pipeline to check the progress.

“dd if=/dev/hda | pv -b | nc -v -l -p 5000″ for example.

And to test if your image is correct, use your good friend md5sum.

md5sum /dev/hda

md5sum /media/sda1/MY_IMAGE_FILE

Remember that you need to be very careful with the dd program…. they don’t call it disk destroyer for nothing… Practice your dd skills on a test system. I’ve clobbered my hard drive before by typing something wrong and there isn’t any undo… good thing I had a backup.

I found this useful to play with the tomsrtbt floppy linux distribution and image an old laptop over the network using tomsrtbt on the old laptop and a 3c589 pcmcia card and knoppix on the other computer. Tomsrtbt doesn’t have the CIFS network protocol, so netcat comes in handy. Tomsrtbt has a slightly different format for the nc command (netcat is also called nc). You don’t need to specify the -p option.

This should get you started, but go check out the netcat page on wikipedia. You’ll find lots of good examples. There are other netcat type programs out there, such as ncat (which is also present on knoppix 7.2 dvd).

If your netcat gets “stuck” (like you forgot the -q 0 parameter) and you want to close it, a simple ctrl-c will make it close the connection.

Parsing YouTube V3 JSON with PHP Example

In this article we’re going over an example of how to parse the JSON output from the YouTube V3 API using PHP.

First you’ll need to login to the Google Developers Console and obtain your API key to use when making queries to the YouTube API.

The following code will execute your query to YouTube and display the title of the video along with a thumbnail and description.

<?

$apikey = “YOURKEY”;
$per_page = 6;
$search = “your search term”;
$category = “2”; //autos

$query = “https://www.googleapis.com/youtube/v3/search?part=snippet&q=$search&maxResults=$per_page&videoCategoryId=$category&safesearch=strict&key=$apikey”;

$json_file3 = file_get_contents(“$query”);

$jfo3 = json_decode($json_file3,true);

foreach($jfo3[‘items’] as $val) {

$title = $val[‘snippet’][‘title’];
$description = $val[‘snippet’][‘description’];
$id = $val[‘id’][‘videoId’];
$thumbnail_url = $val[‘snippet’][‘thumbnails’][‘default’][‘url’];

echo <<<EOF

<p><img width = “250” src = “$thumbnail_url” align = “right”></a>
<a href =”video-viewer.php?v=$id”>$title<BR>
$description</p><br clear=”all”><HR>

EOF;

?>

Downloading knoppix with metalinks

Lots of linux distros and other popular packages have created metalinks for the distribution of their software in a way that failover and redundancy and being able to download files from multiple mirros is supported via simple xml file.
You can take a look at all the distros and packages using metalink at this website
There is a long listing of download managers which support metalinks too.

Looking at the popularity of the metalinks , IETF has approved and it is now a standard.

So I took the opportunity of converting all the mirror information(http/ftp) of Knoppix for version 7.0.5 and have created metalinks for it in both ver3 (extension .metalink) and version 4(approved by IETF as a standard) with file name extension as meta4.
With these metafiles you are limited to only your download bandwidth of your ISP as you can parallely download from multiple mirrors simultaneously and even if some fails it moves to the other in the list.

Pls be sure to read the documentation of your download manager to ensure they support Metalinks and also pls be sure to understand the metalinks which I have published have all the flavours of ISO images(English DVD/CD,German CD/DVD) so pls pay attention when you download and only select the flavor you are interested in.

I have tested that each and every mirror in my metalink file is working at this time.

Pls let me know if there are any bugs etc.Also let me know if you find this useful then I can probably make metalinks to speed up the downloads and spread the load across various mirrors with the upcoming Knoppix 7.1.0 release as well.
Pls note since the Torrent for Knoppix has multiple files in it we cannot add it as a metafile.Currently if you want parallel downloads with HTTP/FTP/BIttorrent simultaneously then the torrent should only have one file which is the ISO file which is on the http/ftp sites.