Sending SMS from terminal –A quick hack

This utility was developed when I was waiting to get no due slip signed by Kasi Raju. A special thanks to him for not being at his desk during office hours. I was always pissed off with the Way2Sms website because it always loads too slow. But on the bright side, the sms sent reaches the receiver almost instantaneously.

The Way2Sms does not provide API to send sms. So we have to think of a alternative way to do it. CURL was the first thing that came to my mind. But i decided to do using wget which is a nifty little tool. I also used a firefox plugin LiveHTTPHeaders to monitor the http request/response made between the browser and the server.

I opened Way2Sms login screen and I entered my username and password. I enabled LiveHTTPHeaders and logged in. All the information exachanged between the browser and server was captured. Analysing many such request/response I found the below one relevant.

So, I crafted a request from the terminal for logging in using wget. I found that I sucessfully logged in from the terminal. This is confirmed by the same page being returned when the request is made from browser and terminal. I used the following command.

//for signing in 

wget  --save-cookies cookie.txt --keep-session-cookies --referer "http://site6.way2sms.com/content/index.html" --user-agent  "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6"  \
--header= "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 " \
--header="Accept-Language: en-us,en;q=0.5 " \
--header="Accept-Encoding: gzip,deflate " \
--header="Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 " \
--header="Keep-Alive: 300 " \
--header="Proxy-Connection: keep-alive " \
--header="Content-Type: application/x-www-form-urlencoded " \
--header="Content-Length: 50" \
--post-data="username=9840281903&password=abcdef&login=Login"  \
http://site6.way2sms.com/auth.cl 
Replace username and password in the command above.

When we successfully login a cookie is returned to the browser. This is sent along with each request to the server. There is also session cookies which is used to store persist information for the entire session. I have given –save-cookies and –keep-session-cookies to store it into a file. I will be using this information when i am sending the sms.
The below image shows the information captured when sending an sms and hence I have crafted a request similar to this using wget.

wget --load-cookies cookie.txt --referer "http://site2.way2sms.com/jsp/InstantSMS.jsp" --user-agent  "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.6) Gecko/2009011912 Firefox/3.0.6"  \
--header= "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 " \
--header="Accept-Language: en-us,en;q=0.5 " \
--header="Accept-Encoding: gzip,deflate " \
--header="Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 " \
--header="Keep-Alive: 300 " \
--header="Proxy-Connection: keep-alive " \
--header="Content-Type: application/x-www-form-urlencoded " \
--header="Content-Length: 155 " \
--post-data="HiddenAction=instantsms&catnamedis=Birthday& \
Action=dsf45asvd5&chkall=on&MobNo=9840281903&textArea=bye& \
guid=username&gpwd=*******&yuid=username&ypwd=*******" \
http://site2.way2sms.com/FirstServletsms 

Enter the your phone number in the command above

Actually the way2sms website is badly designed. It does not require any such headers and works prefectly without this. The trimmed down version is

//To Login

wget  --save-cookies cookie.txt --keep-session-cookies \--post-data="username=9840281903&password=abcdef&login=Login"  \
http://site6.way2sms.com/auth.cl
//To send SMS

wget --load-cookies cookie.txt \  
--post-data="HiddenAction=instantsms&catnamedis=Birthday& \
Action=dsf45asvd5&chkall=on&MobNo=9840281903&textArea=bye& \
guid=username&gpwd=*******&yuid=username&ypwd=*******" \
http://site2.way2sms.com/FirstServletsms 

We can build a small script or a python application which asks the username and password in the terminal and automatically send sms 🙂 🙂 Happy Hacking !!! 🙂 🙂

This is the Pastebin link.

Page Rank

This is a long due post which i wanted to write a year ago after my 1st sem in IIT . The page rank algorithm is a very simple probabilistic model for ranking the importance of page . The page rank (PR) of a website gives the importance of the website in the WEB .

The PR of a page (A) is calculated by the number of inbound links ie number of pages pointing to A . Suppose B , C , D are three web pages having a link to A . Then B,C,D is said to cast a vote of A . But google took another measure into account . It assigned a rank not only based on number of inbound links but also based on the importance of the page which points to A ie. the importance of B , C ,D is taken into account .

The PR of a page is calculated using the below formala
PR(A) = (1-d) + d(PR(t1)/C(t1) + … + PR(tn)/C(tn))

d – Damping ratio. Usually a value of 0.85 is set for this .
t1 – tn – These are the pages which are pointing to A
PR(t1) – The importance of t1 which is pointing to A
C(t1) – The total number of outbound links in page t1 . So if the Page rank of t1 is 0.50 and has 100 outgoing links from t1 then each outgoing link gets 0.50/100 .(uniformly distributed ) .

So the page rank of A is the sum of values contributed by each of it’s incoming links .

Pretty Simple right!!! Google does not take only page rank into account . It uses a number of complex measures . For eg: Suppose you are searching for Bruce Lee . The text processing engine finds web pages with the keyword (Say E is one of the page containing the keyword) . It also looks at pages pointing the page( pages pointing E) and does a context based search of these pages also .

So to in order to increase the page rank of the page one can make manually make a lot of links point to this page . This is called link farming . Google is intelligent enough to identify such link farming website and ignores such links . It also identifies such link farming websites as harmful and if you website points to such penalized pages you page rank will automatically be decreased . One way to increase the rank of ur site is to increase the number of pages in ur website .

A major problem when implementing algorithm is “Problem of Circularity ” . Suppose page A points to B and B points to A . To calculate the PR(A) we need PR(B) and vice versa . The value of P.R(A) and PR(B) calculated are inaccurate . So this procedure of calculating PR is iterated a number of times (ideally 5 times ) so that the value thus calculated is close the real value . This is the reason by PR updation of a webpage takes a lot of time . Google recalculates these values almost every month( not know for sure .. ) .

There was one more reason i am writing this article . I wanted to do some sort of analysis using the emails from my inbox (hobby work) . I thought that page rank algorithm will do the thing . So this is what i wanted to do . I wanted to read all the messages in my google inbox and make each and every address as a node in a graph . I would add an directed edge A–>B if A has received an mail from B . Then applying the page rank algo . I would have information on important nodes in my social network graph . If you have used page rank algorithm for any of the problems kindly post in the comments section .

Tweeting from Python :)

I got this Itch when i was too reluctant to login to twitter to post … It was a pain . I also get pained by manually making Google Calendar entry . Luckily i found that Google has API’s (googlecl) to use it’s services . So i decided to write a Python App which could enable to me to update and check everything from terminal . So decided to start with Twitter , then google services . The problem i faced with Twitter was that it needed OAuth authentication which is serious pain . So i decided to write this tutorial .

First things comes to our mind is .. What is OAuth ??? It is nothing but a Open Authentication protocol which is being widely adopted . To name few Yahoo and Twitter require OAuth autentication to use it’s services . Say you have a twitter account . There is another website X which wants to access your tweets . Then the website X should know your login credentials . In order to over come this OAuth was introduced . With OAuth the website X can be authorized and it can access the twitter account with the access token given to it . The advantage is private information can be accessed by other applications without knowing the password and with only access token .

1) Now to begin with we first need to create Application Key . Visit this website http://dev.twitter.com/apps/new
2) Once registered you will be given with loads of info . Save it carefully . The most important thing in that are Consumer Key and Secret.
3) I am pasting code for Python script which is used to get the Access Token from the consumer key and secret . This a three way handshake process . Don’t worry abt it.. Use the code i’m pasting here . Filling in the details and run the script . You will be given with a URL . Paste the url in the browser and following instructions to get a PIN . Now, paste the PIN in the command prompt . When the script finishes execution you will be given with the access token . Store it safely.

Script for getting Access Token

import urlparse
import oauth2 as oauth

consumer_key='' '  # Fill in your key 
consumer_secret=' ' # Fill in your secret
request_token_url='https://api.twitter.com/oauth/request_token'
access_token_url='https://api.twitter.com/oauth/access_token'
authorize_url='https://api.twitter.com/oauth/authorize'

consumer=oauth.Consumer(consumer_key,consumer_secret)
client=oauth.Client(consumer)
resp, content = client.request(request_token_url, "GET")

if resp['status'] != '200':
    raise Exception("Invalid response %s." % resp['status'])

request_token = dict(urlparse.parse_qsl(content))

print "Request Token:"
print "    - oauth_token        = %s" % request_token['oauth_token']
print "    - oauth_token_secret = %s" % request_token['oauth_token_secret']
print 

# Step 2: Redirect to the provider. Since this is a CLI script we do not 
# redirect. In a web application you would redirect the user to the URL
# below.

print "Go to the following link in your browser:"
print "%s?oauth_token=%s" % (authorize_url, request_token['oauth_token'])
 

# After the user has granted access to you, the consumer, the provider will
# redirect you to whatever URL you have told them to redirect to. You can 
# usually define this in the oauth_callback argument as well.
accepted = 'n'
while accepted.lower() == 'n':
    accepted = raw_input('Have you authorized me? (y/n) ')

oauth_verifier = raw_input('What is the PIN? ')

# Step 3: Once the consumer has redirected the user back to the oauth_callback
# URL you can request the access token the user has approved. You use the 
# request token to sign this request. After this is done you throw away the
# request token and use the access token returned. You should store this 
# access token somewhere safe, like a database, for future use.
token = oauth.Token(request_token['oauth_token'],request_token['oauth_token_secret'])

token.set_verifier(oauth_verifier)
client = oauth.Client(consumer, token)

resp, content = client.request(access_token_url, "POST")
access_token = dict(urlparse.parse_qsl(content))

#print "Access Token:"
print access_token.keys() # Save the output of the script which gives the access token
print access_token.values()
print "You may now access protected resources using the access tokens above." 

4) Once done use the key in ur application . Sample APP


import twitter

api = twitter.Api(consumer_key=' ',
                           consumer_secret=' ',
                            access_token_key=' ',
                            access_token_secret=' ')

friends=api.PostUpdate("First Tweet from PYTHON APP ")

Authenticating with OAuth is the main part . Once it’s done it’s just child play with python .

PS: I have used python-twitter API . Plz dnt download version 0.6 . Only 0.8 version has OAuth support . It’s still under development so it’s best to do a pull from the source .

sudo hg clone https://python-twitter.googlecode.com/hg/ python-twitter
cd python-twitter

Reference counting & Copy on write

The problem when using default copy constructor is if the source contains any pointer types then the destination also points to the same address . So when the source object goes out of scope , deferencing the pointer type from the destination results in Segfault . So that’s why we override the default copy constructor in most cases. Now in high performance application creating a new object in the copy constructor seriously hinders performance . So instead of creating object , the pointer in destination object is just made to point the source object pointer until & unless you want to perform a write action on the object . I’m posting the code which was causing some Segfaults in the destructor and was a pain the ass . Finally it’s corrected . I guess you will enjoy the beauty of c++ and appreciate it’s elegance and power .

#include<iostream>
#include<cstring>

using namespace std;

class Dog
{
	string name;
   int count;

	public:

	Dog(string _name):name(_name),count(1)
	{
	
		cout<<name<<":: The count value is "<<count<<endl;
	}

	Dog ( Dog & ob):name(ob.name),count(1)
	{ 
	
    cout<<name<<":: The count value is "<<count<<endl;	
	}

	void Attach()
	{

		count++;
		cout<<name<<":: The count value is "<<count<<endl;
	}

	void Deattach()
	{
		count -- ;

		cout<<name<<":: The count value is "<<count<<endl;

		if( count ==0)
		{
			cout<<name<<":: Deleted object"<<endl;
			delete this;
		
		}
	}

	Dog*  Alias()
	{
		if( count ==1 )
			return this;
		else
			return new Dog(*this);

	}

	void Setname(string _name)
	{

		name=_name;
	}

};

class Doghouse
{
	Dog *dog;
	string house_name;

	public:

	Doghouse(string _house_name)
	{
		house_name=house_name;
	}

	Doghouse(Dog *_dog, string _house_name):dog(_dog),house_name(_house_name){
	cout<<"The house name is "<<house_name<<endl;
	}
	Doghouse(Doghouse & dh)
	{

		house_name=dh.house_name;
		dog=dh.dog;
		dog->Attach();

	}
	
	Doghouse& operator = (const Doghouse& dh)
	{

		dog->Deattach();
		dog=dh.dog;
		house_name=dh.house_name;
		dog->Attach();
	
	}

	void Alias()
	{
		dog=dog->Alias();
	}

	void rename_dog(string _name)
	{

		Alias();
		dog->Setname(_name);



	}

	~Doghouse()
	{

		dog->Deattach();

	}

	};


int main()
{

	Dog *d1=new Dog("Peggy");
	Dog *d2=new Dog("Hussky");

	Doghouse dh(d1,"kennel");
	Doghouse dh1=dh;
   Doghouse dh2(d2,"kennel2");
	dh2=dh1;

	dh2.rename_dog("husskie");

}
  

Deep Down Inside the Linux Kernel .. Part 2

Lists in Linux kernel

This tutorial will give you a small working code of creating a list which is supposed to be quite difficult coz the kernel is not documented . So even understanding code which used lists and creating your own list becomes quite tedious . So decided to dedicate an article on this one .

The linux kernel has some very efficient implementation of list . It can perform insertion , deletion in O(1) . It’s pretty easy to guess that it’s a double linked list . It is a circularly double linked list . The most brilliant thing about this is it does have a header node . Kernel hackers are proud about this implementation . The reason is that the can traverse the list starting at any node in the list untill the node is reached again . This removes the special code to take care of the corner cases . If you want to write some kernel code which need a list , you should only use the list provided by linux otherwise your code will never be accepted into the mainstream kernel , otherwise you are sure to get strange reaction from the kernel community . I can say the same reaction that you would get if use vmalloc() in kernel code or even worse . So better dnt dare .

The linux implementation of list is in #include “linux/list.h”
You can create a structure using struct list_head structure Eg : struct list_head sample_list;

Internally list_head is implemented as follows
struct list_head
{
struct list_head *next;
struct list_head *prev;
}

This list itself is of no use . You have to embed this structure inside your own structure

For Eg:

typedef struct
{
int data;
struct list_head k_list; //The linux list
}Node;

Next is allocating space and initialisation

Node *node=(Node *)kmalloc(sizeof(Node) , GFP_KERNEL ) ; //Note that this function would sleep . So use it in interupt handlers only when you want to crash your system 🙂

INIT_HEAD_LIST(&node->k_list) ; //To initialse the list

A number of functions are available to manipulate list .But aim is to tell you how to create , initialise list and how to use it which are supposed to be quite non intutive .

To Add a item to the list use
list_add(struct list_head *node , struct list_head *list ) which adds “node” to the right of the “list” .

To iterate through the list use
list_for_each(struct list_head *temp , struct_list_head *list) which is expanded to

for(temp=list->next ; temp !=list ; temp=temp->next ) //Hope you understand this . Kernel provides a macro function to achieve this

The most important take away point is as follows . Take for example our structure Node . The kernel list(k_list) inside our structure points to the next node in the kernel list . It does not point to the next Node . We need to find the address of Node structure to access its variables . We use list_entry() to achieve this

list_entry(temp,Node,k_list ) where temp is the address of the list item in the Node structure . Node is the type of the structure .k_list is the name of the list inside our structure .

I wrote a sample code to create a list which i though would be handy for others .

#include “linux/kernel.h”
#include “linux/module.h”
#include “linux/init.h”
#include “linux/sched.h”
#include “linux/list.h”
#include “linux/slab.h”
typedef struct
{
int data;
struct list_head list;
}Node;

int abhi_init(void)
{
int i;
Node *node;
struct list_head *head=NULL,*Next=NULL;
for(i=0;idata=i;

INIT_LIST_HEAD(&node->list);

if(head ==NULL)
{
head=&node->list;
Next=head;

}
else
{
list_add(&node->list,Next);
Next=Next->next;

}

}

printk(“\nPrinting data”);
printk(“\nThe value is %d ” , (list_entry(head,Node,list))->data );

list_for_each(Next,head)
{
node=list_entry(Next,Node,list);
printk(“\nThe value is %d “,node->data);

}

return 0;
}

void abhi_exit(void)
{

printk(“\nModule Exiting “);
}

module_init(abhi_init);
module_exit(abhi_exit);
MODULE_LICENSE(“GPL”);

Happy Kernel Hacking!!!!!!!!!!

Deep down inside the linux kernel .. Part 1

Process Management

After a very busy 2nd semester finally decided to dedicate an article on Process Management .

What can readers expect in this article ?????/.

1)A comphrehensive info on process and its management

2)The beauty and simplicity of process management unlike windows OMG!!!! . Windows in the name of abstraction have complicated so many things .

In linux a task is represented by struct task_struct which declaration is present in sched.h . Use cscope utility to search through the kernel . cscope coupled with vim is like a soldier armed with a sword and shield .

Linux does not have the concept of threads . There is no difference between threads and process . Lets analyse the need for a thread . A process can have multiple paths of execution(each execution path is a thread ) . To speak in the language of computers . Each process has a process descriptor with a pid . The process descriptor points to threads . Each thread access the resource of the process and also can have resources private to it. Thread is light weight process (the time to make a thread ready to run or context switch is very less ) .

Linux philosophy is that process in linux are itself light then why should have something called a thread . So in linux when you want to create threads (For eg: 4 threads) we create 4 task_struct structures . They are created using the clone system call ( clone() system call is internally called by fork() , vfork() ).

clone( CLONE_VM | CLONE_FS | CLONE_FILES |CLONE_SIGHAND ,0 )

CLONE_VM – The address space of the parent and child should be shared
CLONE_FS – The file system resources should be shared
CLONE_FILES – The file descriptors(including the device files ) should be shared
CLONE_SIGHAND – The signal handlers should be shared

When a parent creates 4 childs and each will share the address space of the parent and each thread can have private resources and they are light weight which is nothing but a thread .

Everyone would know fork() ( remember the small code snippet in Galvin 😉 ) Then what is vfork() ??? Why we need them ???

When we use vfork() the parent suspends its execution and allows the child to execute in the address space of the parent . Any change made ny child will be visible to parent . vfork() was introduced as an alternative because of the overhead involved in initial implementation of fork() which duplicates the page tables of parent for the child . Then fork() was reimplemented to use copy_on_write() so vfork() is seldom is nowadays . vfork() is also implemented using clone

clone(CLONE_SIGHAND |CLONE_VFORK , 0)

Process Termination

It is one of the most interestesting and mysterious topic one would face . Ask a person what a zombie process is ?? Each would come up with a truely different answer . The best way to clear the doubts is go through some Linux kernel books and by investigating the kernel code

What happens when process exits ??

1) The process exits by calling exit() function . When the process’s state is set to TASK_INTERUPTTIBLE then it will respond to external signals . In such a case also when it receives a command to terminate itself it will call the exit() function . From the exit() function the control moves to do_exit() which does a huge number of tasks

2)It will call del_timer_sync() to remove any timers the process has queued in the timer list .

3)It will deallocate the mm_struct(used for process management ) if it is not shared . The function which does is exit_mm() .

4)If the process is enqueued in the wait queue for wait on any semaphore it is dequeued . The function which does is exit_sem() .

5)The exit_fs() , exit_files() , exit_namespace() , exit_sighand() decrements the usage count for the resource which the child held.

6)Then after this it calls exit_notify() which called the forget_orginal_parent() function to reparent the child of the process to init process(has pid 1) or another process in the thread group . After this the state of the process is set to TASK_ZOMBIE and it next step is to call the schedule() function .

7)When the process is in the zombie state. All the resources held by the process is removed . All it holds it the thread_union structure ie . It holds the kernel stack , thread_info structure , task_struct structure . It holds 8kb worth space . The task_struct is not deallocated because it contains the exit code and status of the child which will be read by the parent . When the parent issues wait() system call the child process status and return code is read and the zombie process is removed by calling the release_task() . Again it performs a number of steps which you can finding by investigating the kernel code of exit.c source file .

8)There is no risk of zombie process because each and every child of the parent is reparent to init . init calls the wait() system call for removing the zombie process

Note : There is a difference between orphan process and zombie process . An orphan process is one which has lost its parent and reparented to the init task.

This is some info on Process Management . You can say just an introduction .I’m planning to write more when i find sometime . Till then c u ppl!!!!!

Semaphore Vs Spinlocking

It’s been quite a long time since i updated the blog … Not because i was busy but just because i was too lazy or may be i can say i dint find anything useful to write .

Anyway during the device driver class we had a discussion on implementing Reader/Writer problem in the kernel space to understand wait queues and locking . We started realizing that locking is not as easy as it sounds or as easy as it appeared for those who studied these concepts from galvin . During the demo a open question was posted as to use semaphore or a spin lock for ensuring mutual exclusion for the reader/writer problem . So this tutorial is for those who thinks semaphore mechanism and spin locking are the same.

In a semaphore when we call the down(struct semaphore *) or down_interruptible(struct semaphore*) is that, when the shared resource is locked ( Note : Rem that the shared buffer cannot be locked only instructions that access can be locked) the down call will put the calling process to sleep . So can this mechanism be applied for all synchronization . A big NO!!!!!!!!!! . Suppose we r using a semaphore to sync access to a device in a interrupt handler . Can u put the interrupt handler process to sleep ??? The interrupt handler will only be wakened after quite a long time . I think you should be able to guess the consequences . So what do we use here ???? It’s where the Spinlock kicks in…

The spin lock cannot be put into sleep . It will continuously poll for the processor and it will can be used in process like interrupt handlers . So know with this knowledge which one do we use for Reader/Write problem ???? Take this scenario : Initially the circular buffer is empty and the reader tries to read the data from the buffer . It gets the lock and tries to read from the queue . If it is a spinlock it will not relinquish the processor . So writer process cannot even write the data into buffer . So wat happenns the kernel hangs . The only way is switch of the PC DIRECTLY . No other way . So u must understand how sensitive these issues are and how tough it is to write such high privileged code .

FOOD FOR THOUGHT :
1) We know that the 2.4 is non preemptive .Only one kernel process/thread can run . Then why use synchronization constructs .
2) What is the effect of interrupts in spinlocking ???

Chao!!!!