Friday, June 27, 2014

Transferring images through WebSockets as text with Base64

While communicating through WebSockets, there are various kinds of data we might transfer between servers and clients. These data can be mainly classified into two categories as text data and binary data. When we are sending some strings of messages which can be printed in textual format as it is, they are text data. We have to deal with binary data when we have things such as files, multimedia, etc to be transferred between WebSocket clients and servers. The focus of this blog post is to share some information which I explored recently. Its about transferring some binary data through WebSockets however not as binary data but instead as textual data. How exactly we do such a thing. This is where Base64 encoding/decoding fits in.

Base64 is a way to encode binary data as ASCII text strings so that we can transfer them through channels which are specifically designed to transfer texts. For example, a WebSocket client can take an image file, encode it in Base64 and transfer it as a text stream through a WebSocket to a server. The server can decode the text stream and regenerate the binary patterns that resembles the image file at server side. That is the scenario which I will show with example codes in this article. I will leave the further readings about Base64 up to interested parties. There are lot of on-line articles to read about it and a good place to understand it simply is this Wikipedia article.

Now it's time to try some example codes. This example assumes that you have installed and tried Autobahn WebSocket library in your Linux box already. If not, checkout this article which I wrote a couple of weeks back. Once you have settled everything with Autobahn installation and tried the echo client / server example, we are ready to begin this exploration. We need three things to try a file transferring scenario between a server and a client through a WebSocket. Those are, a client program, a server program and a file to be transferred. We will write the server and client programs in Python language using the Autobahn Python library. The source codes shown below can also be downloaded from GitHub.

Client program code ( client.py )

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
from autobahn.twisted.websocket import WebSocketClientProtocol, \
                                       WebSocketClientFactory

# import the base64 module
import base64

class MyClientProtocol(WebSocketClientProtocol):

   def onConnect(self, response):
      print("Server connected: {0}".format(response.peer))

   def onOpen(self):
      print("WebSocket connection open.")

      def hello():

         # opening the image file and encoding in base64
         with open("image.jpg", "rb") as image_file:
            encoded_string = base64.b64encode(image_file.read())

         # printing the size of the encoded image which is sent
         print("Encoded size of the sent image: {0} bytes".format(len(encoded_string)))

         # sending the encoded image
         self.sendMessage(encoded_string.encode('utf8'))

      hello()

   def onMessage(self, payload, isBinary):
      if isBinary:
         print("Binary message received: {0} bytes".format(len(payload)))
      else:
         # printing the size of the encoded image which is received
         print("Encoded size of the received image: {0} bytes".format(len(payload)))

   def onClose(self, wasClean, code, reason):
      print("WebSocket connection closed: {0}".format(reason))



if __name__ == '__main__':

   import sys

   from twisted.python import log
   from twisted.internet import reactor

   log.startLogging(sys.stdout)

   factory = WebSocketClientFactory("ws://localhost:9000", debug = False)
   factory.protocol = MyClientProtocol

   reactor.connectTCP("127.0.0.1", 9000, factory)
   reactor.run()


Server program code ( server.py )

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
from autobahn.twisted.websocket import WebSocketServerProtocol, \
                                       WebSocketServerFactory

# import the base64 module
import base64

class MyServerProtocol(WebSocketServerProtocol):

   def onConnect(self, request):
      print("Client connecting: {0}".format(request.peer))

   def onOpen(self):
      print("WebSocket connection open.")

   def onMessage(self, payload, isBinary):
      if isBinary:
         print("Binary message received: {0} bytes".format(len(payload)))
      else:
         print("Text message received, saving to a file")         

         # decode the image and save locally
         with open("image_received.jpg", "wb") as image_file:
            image_file.write(base64.b64decode(payload))

      # echo back message verbatim
      self.sendMessage(payload, isBinary)

   def onClose(self, wasClean, code, reason):
      print("WebSocket connection closed: {0}".format(reason))



if __name__ == '__main__':

   import sys

   from twisted.python import log
   from twisted.internet import reactor

   log.startLogging(sys.stdout)

   factory = WebSocketServerFactory("ws://localhost:9000", debug = False)
   factory.protocol = MyServerProtocol

   reactor.listenTCP(9000, factory)
   reactor.run()

As you can see from the client program code, we need to have the image.jpg file which is supposed to be transferred from the client, in the same directory as client.py file. Let's run the programs and see how they work. Open two terminals and run the server and client programs in those two separate terminals. Server will create a new image file named as image_received.jpg based on the data stream it received through the WebSocket.

python server.py

python client.py

This simple example demonstrate how we can transmit binary data such as image files through WebSockets as textual data. Even though we are allowed to transmit binary data through WebSockets as it is, there can be practical scenarios where we have to restrict the communication to text only. In such situations, Base64 can play a great role to assist us on encoding and decoding binary data with textual data.

~***********~

Wednesday, June 25, 2014

My 'Contact' with Carl Sagan

Few months ago, I finished reading an important book. Today, I suddenly decided to leave a little note about it to honor the book and the person who gave it to me. This is not a review of the book as it may appear depending on the way people look at it. First it's necessary to mention how it came onto my hand. Nearly ten months ago while I was in South Korea, a 'very special best friend' gave a book as a gift on my twenty sixth birthday. Even though I was really looking for a chance to sit down peacefully in a fine evening and start reading till the end, I was not able to do that. Those days were a period full of troubles and challenges which pushed me to worry about the survival instead of reading a novel in a calm mind. Therefore, I just read page by page for many months until the end of March this year, I return home. After returning to home, I read it straightforwardly till the end. Lot of things have changed since the day I receive this book from my 'special best friend. Now she is not only a 'very special best friend', but also my fiancée. Quite surprisingly, I found a person who love to read just like me.

Now, this is about the book. The novel she gifted me on that day is 'Contact' written by the great astronomer and writer Carl Sagan. I was a fan of Sagan for few years because of his 'Cosmos' TV series however, I was very unlucky for not getting in contact with any of the books he had written until recently. Finally, 'Contact' book came onto my hands. As a science fiction reader, my only favorite writer was Arthur C. Clarke due to many of his books which I love such as 'Fountains of Paradise', 'Profiles of the Future' and 'Rendezvous with Rama'. Reading 'Contact' helped me to place Carl Sagan into my favorites list as a science fiction writer. In the initial days, I was fascinated by the way Sagan speak and explain things verbally. I really enjoyed watching his programs because of his poetic way of explaining scientific facts. He is a great example to show how much it is amazing and beautiful when a scientist is a great poet. That respect I had on him got doubled right after I got to read 'Contact'.

I would like to windup this little note with one of the thousands of great quotes from Carl Sagan. This one is interesting here because its about books (quote is extracted from Wikiquote). I hope I will be able to read many more of his books. This is just the beginning.

"What an astonishing thing a book is. It's a flat object made from a tree with flexible parts on which are imprinted lots of funny dark squiggles. But one glance at it and you're inside the mind of another person, maybe somebody dead for thousands of years. Across the millennia, an author is speaking clearly and silently inside your head, directly to you. Writing is perhaps the greatest of human inventions, binding together people who never knew each other, citizens of distant epochs. Books break the shackles of time. A book is proof that humans are capable of working magic."
- Carl Sagan (Cosmos: A Personal Voyage)

~***********~

Saturday, June 21, 2014

Creating a Wifi Hotspot in a Ubuntu 12.04 LTS System for an Android device

A huge requirement I had sometime back is allowing my Android device to access Internet by using the connectivity of my laptop. I use a HSPA dongle connected to my Ubuntu 12.04 LTS system via USB port for connecting to the Internet. Additionally, my Android device has Internet connectivity through the mobile network however is has a limited data capacity compared to my laptops' Internet connection. So, what I really wanted do is, spare my mobile data connection in the Android device and use Internet via my laptop as much as possible specially when I need to do some heavy work such as installing some big app or updating the Android system and apps.

The solution is making a Wi-Fi access point in my laptop so that my Android device can connect to the Wi-Fi network to access Internet. After searching in the web and trying different ways, I found two methods to do it. Even through the both methods are supposed to work properly, only one worked for me. I will describe the worked method for me and also at the end I will mentioned the other method which didn't work.

Method - 1

First, you should download the shell script given in this link. Save it to somewhere in your system such as Downloads directory. We should run the shell script from the terminal as follows and also the other commands to start and stop the access point when needed.

sudo sh install_wifi_access_point.sh

sudo service wifi_access_point start

sudo service wifi_access_point stop

Each time  I start the service, the network manager of Ubuntu gets restarted and therefore I had to reconnect to the Internet connection of my laptop using the HSPA dongle. Anyway, its not a big issue.


Method - 2

This is the method which didn't work for me. However, hoping that somebody will get benefited from it, I'm sharing that information too. In this method, we need to install a package called ap-hotspot. Do it using the terminal as follows.

sudo add-apt-repository ppa:nilarimogard/webupd8
 
sudo apt-get update
 
sudo apt-get install ap-hotspot

We can configure the ap-hotspot tool for the first time using the following first command. After that, we can use the other two commands to start and stop the Wi-fi hotspot.

sudo ap-hotspot configure

sudo ap-hotspot start
 
sudo ap-hotspot stop

Hope this information will be useful for somebody in the future including me too because definitely I will forget how I did it after a while. :)

Wednesday, June 11, 2014

Playing with WebSockets

WebSockets is a great way to build client-server applications where full-duplex communication is possible. That means, once a WebSocket client is connected to a WebSocket server, the server gets the capability of pushing messages to the client side without employing traditional hacks such as AJAX and long-polling at the client side. WebSockets specification has started as a part of HTML5 and now its running independently as I know. 

There is a nice implementation of WebSockets as a library which I tried today. It is Autobahn project which provides implementations of WebSockets for various platforms and languages such as Python, JavaScript, C++ and Android/Java. As the first step of exploring the capabilities and features of Autobahn library, today I ran the echo client / server example of the Autobahn project. Here are the basic steps I followed. 

The echo server example of the Autobahn project is in its Python based project while I'm going to use the Android based echo client for communicating with that server. First of all we install the Python based Autobahn project on the Ubuntu 12.04 system. Goto the AutobahnPython project in Github, take a clone of the source code, do some installation stuff and at the end, run the sample echo server.

git clone https://github.com/tavendo/AutobahnPython.git

cd AutobahnPython
git checkout v0.8.5

cd autobahn
sudo python setup.py install

cd ../AutobahnPython/examples/twisted/websocket/echo
python server.py

Now, the echo server must be up and running. It's time to prepare the client for Android. Goto the AutobahnAndroid project in Github and take a clone of the source code as follows.

git clone https://github.com/tavendo/AutobahnAndroid.git

Open the Eclipse IDE (which you must have already configure with Android SDK tools) and select the Autobahn directory from the import project window. It will show many example projects together with a project called autobahn are ready to be imported. Import them to the workspace. Among other example projects, EchoClient project is what we need. If Eclipse complain about some missing build paths and stuff, fix those carefully. In my case, two library Jar files were missing. So, I right-clicked on EchoClient project and took the properties option. In the Properties window, I went to the Java build path option and from there, went to the Libraries tab. Two Jar files were shown as missing. I selected each one and then clicked the Edit button. There I could browse through the autobahn project and selected the relevant missing Jar files. Clicked OK and then things were fine. 


Now I connected my Android device which runs Android 4.3 and then using Eclipse, I built and installed EchoClient app on the device. It's time to turn the client on. I need my Ubuntu machine and Android device to be in the same wireless LAN for this task. So, I created a WiFi access point on Ubuntu machine and connected the Android device to the access point. Now, in the EchoClient app on the Android device, I had to input the IP address of the Ubuntu machine where the echo server runs and click the Connect button. Once they are connected, type a message to be sent from the client to the server and click the Send button.

The message should be sent to the echo server and then echoed back to the client which will be displaced on the Android device as a pop-up message. First figure shown is the output of the echo server on the terminal of the Ubuntu machine while the second figure depicts a screenshot of the EchoClient app.

This is the beginning of the exploration of WebSockets for me.  Lets see what we come across in the future.