How to extract a gzip file or ungzip in Mac OS via terminal

Hello, everyone. Today, I was trying to extract or ungzip a file compressed using gzip compression on Mac OS. Unfortunately, I couldn’t do it by extracting, as there was no predefined UI option that appeared after right-clicking on the file.

Gzip compression comes into play to compress HTTP content before delivering it to the client. Hence, gzip has become popular nowadays. If you are using nginx for your servers, you might have seen “gzip on” in your nginx.conf file.

Let’s address our problem of extracting a .gz file. I have to do it the old way, the “Terminal Way (which I always prefer),” to extract the gzipped file.

To Extract a gz file:

gzip -d path-to-your-file.gz

Here, -d stands for uncompress/decompress.

To compress a file using gzip compression, use the following command:

gzip -k <path-to-your-file>

Here, -k stands for compress and keep the original file. If you don’t want the original file, then use:

gzip <path-to-your-file>

Please note that the above command will modify your original file, so be aware of it while using it without any options.

To test a compressed file, use the following command:

We can verify our compressed file by using the following command

gzip -t <path-to-your-file> -v

-t stands for test, and -v stands for verbose. This will return output like:

<your-file-name>:   OK

So far, we have seen how to compress and decompress gzip files. Can we gzip a directory using gzip alone? Unfortunately, you can’t compress a directory using gzip. Don’t worry; there is a hack for that too.

To compress a directory using gzip, when you try to compress a directory using -r with gzip, it compresses all files inside a directory with gzip compression.

For example, if you have a directory structure like this:

home
   |
   test/
        -- fileone.csv
        -- filetwo.csv
        nestedtestdir/
            -- nsfilefileone.xlsx
            -- hello.txt

What happens when you fire this command:

After this, the directory structure becomes:

gzip -r test
test/
   -- fileone.csv.gz
   -- filetwo.csv.gz
   nestedtestdir/
      -- nsfilefileone.xlsx.gz
      -- hello.txt.gz

As you see, it recursively goes to each file inside the folder and compresses that file, saving it with a .gz extension.

Note: It modifies all the original files inside that folder and also other folders inside the given folder. In case you don’t want to modify the original files, add -k to keep the original files inside a directory like this:

gzip -r <your-directory> -k

To compress a directory using gzip without compressing files inside it, the hack would be to use tar and pass the tar file to be gzipped.

tar zcvf <your-gzip-file-name>.tar.gz <dir-name-to-be-gzip>/

Here, zcvf stands for z -> compress archive with, c -> create, v -> verbose, f -> filename.

You can achieve the same by:

tar -cvf - your-dir-name | gzip > your-dir-name.tar.gz

Here, -cvf stands for compress, verbose, filename after hyphen (-) your directory name. The | (pipe) symbol is used to pass on the result from one command to another command, then gzip is used to compress, and > (greater than) symbol denotes writing into your-dir-name.tar.gz, the target/output file name with .gz extension.

That’s it. Thanks for reading! Feedback is always welcome. Happy times !!!

How to get video files duration and resolution in python

Hi all,

Today i was about to find video file duration and resolution and few more info in python.

In debain based systems type this command in terminal

avconv

If this shows version , like this

avconv version 9.18-6:9.18-0ubuntu0.14.04.1, Copyright (c) 2000-2014 the Libav developers
built on Mar 16 2015 13:19:10 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
Hyper fast Audio and Video encoder
usage: avconv [options] [[infile options] -i infile]… {[outfile options] outfile}…

Use -h to get full help or, even better, run ‘man avconv’

If it says commond or package not found, then install this by

sudo apt-get install libav-tools

After successful installation, In terminal

avconv -i "filepath"  # give path to your video file

You will see output like this
Metadata:
    major_brand     : mp42
    minor_version   : 1
    compatible_brands: mp42mp41
    creation_time   : 2015-10-21 16:41:08
  Duration: 00:00:30.00, start: 0.000000, bitrate: 748 kb/s
    Stream #0.0(eng): Video: h264 (Constrained Baseline), yuv420p, 480×240, 488 kb/s, 25 fps, 25 tbr, 25 tbn
    Metadata:
      creation_time   : 2015-10-21 16:41:08
    Stream #0.1(eng): Audio: aac, 48000 Hz, stereo, fltp, 255 kb/s
    Metadata:
      creation_time   : 2015-10-21 16:41:08

You can see the resolution and duration highlighted my result.

Now lets see how to use this in python to get video resolution, duration.

The idea is simple , am gonna use subprocess and pipe the output

Here is the python code.


from subprocess import Popen, PIPE
import re

def getvideodetails(filepath):
    cmd = "avconv -i %s" % filepath
    p = Popen(cmd, shell=True, stdout=PIPE, stderr=PIPE)
    di = p.communicate()
    for line in di:
        if line.rfind("Duration") > 0:
            duration = re.findall("Duration: (\d+:\d+:[\d.]+)", line)[0]
        if line.rfind("Video") > 0:
            resolution = re.findall("(\d+x\d+)", line)[0]
    return duration,resolution

# call function with file path
getvideodetails("filepath") 

This code can be found in gist too here

Thats it.

Happy coding!!!
Happy times !!!

MongoDB Basics

Hi all. Today we are going to see about mongodb basics.

what is it?

Mongodb is a No-Sql Database , document based database (closely dictionary in python , ruby).  Here you can read about it official documentation .

why is it used for?

It’s used when you no need to care about your schema . It means when you are using mysql , we have to have ID field integer, name varchar . Here it’s not needed .

Installation:

open terminal (cntl+alt + T)

sudo apt-key adv –keyserver hkp://keyserver.ubuntu.com:80 –recv 7F0CEB10

sudo apt-get update

sudo apt-get install -y mongodb-org

Hope you have successfully installed mongo in your system.

if you’re having trouble see official site for trouble shooting here

Time to play with it :

type mongo and enter in your shell

you will see something like this

MongoDB shell version: 2.4.9
connecting to: test

>

To see all database:

show dbs 

it lists all databases. This is equal to show databases in MYSQL

To select a particular database

use databasename

To see all collections inside a database.

Note: In mongodb collections refers tables. we won’t call them as tables . It’s collections here.

show collections

this is equals to show tables in MYSQL

Let’s create a database called bala .

use bala

unlike MYSQL it won’t throw you error unknown database.

to check in which db you’re issue db command  in mongo shell.

it shows bala

now create a document in it.

db.mycollection.insert({“name”: “bala”})

what i did here?

db => bala

mycollection => our collection name

insert => operation name

{“name”: “bala”}  => data we are inserting into our collection

To see a document is inserted into our db , type this command

> db.mycollection.find()
{ “_id” : ObjectId(“55a6070cab3e0cf5c31434cf”), “name” : “bala” }
>

see our document is inserted, find returns all documents in a collection(table) .

Have you noticed _id in above output. Yes that’s automatically created by mongo . Its alpha numeric . Similar to id primary key in MYSQL. 

To see one document in a collection

> db.mycollection.findOne()
{ “_id” : ObjectId(“55a6070cab3e0cf5c31434cf”), “name” : “bala” }
>

Now try inserting other document

db.mycollection.insert({“name”: “hari”, “country”: “india”, “state”: “Haryana”, “age”: 20})

what we did ?

we gave country , state, haryana . so many keys .  what happened?

As i said it’s schema less database , it doesn’t care about schema.

now let’s go and check it out.

> db.mycollection.find()
{ “_id” : ObjectId(“55a6070cab3e0cf5c31434cf”), “name” : “bala” }
{ “_id” : ObjectId(“55a60976ab3e0cf5c31434d0”), “name” : “hari”, “country” : “india”, “state” : “Haryana”, “age” : 20 }
>

see it shows two documents in our collection. how cool is this .

To find a document by condition (by name , age or any field)

let’s see i want to see all document’s which have name “hari”

> db.mycollection.find({“name”: “hari”})
{ “_id” : ObjectId(“55a60976ab3e0cf5c31434d0”), “name” : “hari”, “country” : “india”, “state” : “Haryana”, “age” : 20 }
>

here i have only one document having name “hari”, if I ‘ve many then it will show all documents.

We can see particular field value also by condition

let’s say i want to see all names alone in my collection

> db.mycollection.find({}, {“name”:1})
{ “_id” : ObjectId(“55a6070cab3e0cf5c31434cf”), “name” : “bala” }
{ “_id” : ObjectId(“55a60976ab3e0cf5c31434d0”), “name” : “hari” }
>

{} => empty dict to select all documents

“name”:1 => select only name from all documents

we can select  as many documents as possible

fieldname: 1  => selects the field to result.

fieldname:0  => ignores the filed to result.

Here instead of { } , we can give condition also .

Try it out those commands also.

Let’s update a document named “bala” , adding age field into it

> db.mycollection.update({“name”: “bala”}, {$set : {“age”: 20}})

$set is used to update field

let’s see it’s updated or not.

> db.mycollection.find({“name”: “bala”})
{ “_id” : ObjectId(“55a6070cab3e0cf5c31434cf”), “age” : 20, “name” : “bala” }
>

its updated , age field added into it. if we have many documents named “bala” then all got updated by above command.

till now we seen creation,insertion, updation.

Let’s move on to deletion of a document

To delete a document in collection

> db.mycollection.remove({“name”: “bala”})
> db.mycollection.find({“name”: “bala”})
>

first command deleted the document, second one verifies that we don’t have document name “bala”.

To remove all documents in a collection

db.mycollection.remove()

This command removes all document in a collection.

That’s all about mongodb basic (curd operations) .  Try playing with that , we will see with advanced commands or integration with any language (python or ruby) later .

 

Thanks for reading . Happy hacking .

 

 

 

Nginx to serve static files example

Hi all .. Today we ll see how to use nginx to serve static files .

what is Nginx?

Niginx is a high performence web server software. Its more flexible and lightweight than apache. Niginx is a reverse proxy protocol.

wait a minute.. what is meant by reverse proxy ?

lets say x- a client and y –  a  server

x->y

x sends request to server [this is direct proxy]

in reverse proxy

x sends request to server but request is sent to some internal proxy and that proxy passes request to server and sends response from server to the client. The client may not be aware of this.

In short, requests are not directly processed by server some intermediate guy will process requests and responses.

x->z->y   [here z is that intermediate guy]

Got something ??

Lets see the installation of nginx

1. open Terminal (cntrl + alt +T)

2. sudo apt-get install nginx

Lets play with nginx now .

To play ,lets create one directory

mkdir /cache

cd /cache/

chmod 777 /cache/   

lets create some dummy files in this directory

 cat >>hi.txt

hi this is test file

this is line

cntrl +  d  [press contrl + d to save this file]

cat >> second.txt

this is my second file

this is testing second file

press contrl + d to save this file .

Now ,its time create a sample configuration file to server this directory.

Before we write ,its useful  to know about two import directory in nginx

1. sites-enabled

cd /etc/nginx/sites-enabled/  

This directory contains symlink files from sites-available directory

[we can create files here but don’t do that it’s not a good way to do]

2. sites-available

cd /etc/nginx/sites-available/

here all the config files are there for nginx

make sure your inside sites-available directory else

do cd /etc/nginx/sites-available/

touch sample.conf

open this sample.conf in your favorite  editor  and type the below code

server {
    listen                *:9000;
    server_name          _;
    access_log            /var/log/sample.access.log;  # this line is optional for log purpose
    error_log             /var/log/sample.error.log;  # this line is optional

    location /test/ {
        alias /cache/;
        autoindex on; 
        expires off;
        default_type application/octet-stream;
   }
}

Lets see ,

listen *:9000  –> server listens to the port 9000

server name  – is optinal [you can give your domain name like http://www.example.com,]

next two lines are to log who accesses files and error details

next loctaion /test/{

telling nginx -x to server this location

alias  /cache/     —> /cache directory can  be accessed using /test/

auto index on —> to list all the files inside directory [ not recommended ,because user will know all the files inside your directory ]

expires off  [ this disables caching files ]

default_type application/octet-stream   –> this sets ,whenever the user visits this page it serves the content of the file .[asks for open or download]. It never opens in browser

if you want to just view in browser not to download

just comment that last line which contains octet-stream

just save this file now

now lets create a symlink to sites-enable

It will be like this

sudo ln -s   /etc/nginx/sites-available/sample.conf    /etc/nginx/sites-enabled/

Note: Always use absolute path . don’t use relative path like this sudo ln -s sample.conf /etc/nginx/sites-enabled/  — this will not work . Always use absolute path

ok lets restart nginx server

sudo service nginx restart

open your favorite browser and visit localhost:9000/test/

you will see like this

nginx auto index on

click on the any file [ note i commented octet-stream]

content view in browser

 

if you want file to be downloaded uncomment that last line . You browser will ask file to be opened or download

 

Thats it … Play with it .. Happy coding..:)

 

 

 

 

 

 

 

 

 

 

 

 

How to format pendrive or Memorycard or USB in ubuntu by Terminal

Hi…all today ..i tried to to format by pendrive by our file manager. But sometimes it fails to format.So we have to format it by terminal.

After plugging your device(usb or pendrive or memory card).

Open Terminal(cntrl +Alt + T)

type the command as $ dmesg | Tail

it will display like this

[ 1749.108796] sd 9:0:0:0: [sdb] Write Protect is off
[ 1749.108813] sd 9:0:0:0: [sdb] Mode Sense: 43 00 00 00
[ 1749.109803] sd 9:0:0:0: [sdb] No Caching mode page present
[ 1749.109816] sd 9:0:0:0: [sdb] Assuming drive cache: write through
[ 1749.116895] sd 9:0:0:0: [sdb] No Caching mode page present
[ 1749.116906] sd 9:0:0:0: [sdb] Assuming drive cache: write through
[ 1749.118911]  sdb: sdb1    –> Things to be noted
[ 1749.122785] sd 9:0:0:0: [sdb] No Caching mode page present
[ 1749.122800] sd 9:0:0:0: [sdb] Assuming drive cache: write through
[ 1749.123771] sd 9:0:0:0: [sdb] Attached SCSI removable disk

Then Unmount your device by

$ sudo umount /dev/sdb1

Finally Enter this command to format Your device

$ sudo mkfs.vfat -n ‘Ubuntu’ -I /dev/sdb1

Thats it….Now plug out your device and plug in again to use . Your device is now formatted and ready to use now..

Happy coding!!!!

How to fix subtitles delay or ealier with your movies by python code

Hi..all …today i was watching movie with subtitles . I had delay with my subtitles files ,like  subtitles mismatch with  the every scenes of movie.  .

I sloved this issue by python code .Subtitles are in .srt format. (e.x) Pirates of the Caribbean -The Curse of the Black Pearl(2003).srt

I found that i got delay by 2 minutes in my .srt file

Library that  i used : pysrt

Open your terminal (cntrl+alt+t)

type python —> enter into python interpreter mode

import pysrt

if you got any error like –> no moduled named pysrt.  then you need to install pysrt .To install pysrt – sudo easy_install pysrt

for python3 users its available pysrt3

if don’t get any error then you already have this library.

now , if you want to make a delay in .srt file means do like this

 

>>> subs.shift(seconds=-2) # Move all subs 2 seconds earlier
>>> subs.shift(minutes=1)  # Move all subs 1 minutes later

finally save this file from your terminal by

>>> subs.save('path to ur location/newfilename.srt', encoding='utf-8')

My Entire code which sloved my delay in .srt file

#! usr/bin/python
import pysrt
subs=open("/home/bala/Pirates of the Caribbean -The Curse of the Black Pearl(2003).srt")
subs.shift(minutes=-2) # Move all subs 2 minutes earlier
subs.save('/home/bala/new.srt', encoding='utf-8')#saves file with new.srt in your home directory

This sloved my problem .
Thats it..   Happy coding !!!!