Category Archives: How-to

[HOW-TO] Creating a REST webservice with Python and Bottle

Posted on by
Reply

Nowadays there are mutiple web applications that need to provide an API to let the rest of the world interact with its services. One approach to achieve a fast, easy-to-develop and easy-to-use API is using REST.  If you don’t know what is REST about, I recommend you this video: An introduction to REST.

This time I am using Python to develop the example. We are going to need a python web framework called Bottle which makes the creation of the REST service a piece of cake. Bottle runs with Python 2.5+ and 3.x, make sure you are using a supported version. You can download Bottle from PyPI or install it via:

pip install bottle

Code is very straight-forward. We need to import bottle route, run and request. Each time we want to map a request, all we need is to define @route(‘/map’, method=’REST_METHOD’) above the method that will manage that request. For example: @route(‘/helloworld’, method=’GET’). That would take account of /helloworld and will generate a response for that request. Also, Bottle framework provides a method to wait and attend the requests: run(). It is configurable (host, port, server name…) but if you call it without parameters it will start a server listening to localhost:8080.

As a response, we should provide the requested information in JSON or XML format. Python has libraries to make this duty easy. These formats are used because they are easy to parse.

Bottle framework has capabilities to read from Cookies, HTTP Headers and Query Variables as well. There is more information about in http://bottlepy.org/docs/dev/tutorial.html. Here, there is a little example to create the REST API:

'''
Created on 26/06/2012
@author: Héctor Veiga. hveiga.com
'''
import bottle
from bottle import route,run,request

@route('/helloworld', method='GET')
def hello():
    return 'Hello World!'

 # JSON format response
@route('/book/:id', method='GET')
def book():

    return {'id':id, 'name':'A Game of Thrones: A Song of Ice and Fire'}

# Query string is /action?id=4&op=6. JSON format response
@route('/action')
def select_action():

    action_id = request.query.id
    option = request.query.op

    return {'action_id':action_id, 'option':option, 'result':'200 OK'}

# Image there is a text input named 'title' and a file input name 'image'
@route('/upImage', method='POST')
def upload_image():
    title = request.forms.title
    image = request.files.image
    if title and image and image.file:
        raw_data = image.file.read()
        filename = image.filename
        return 'Image uploaded!'
    return 'Something went wrong. Please try again'

[HOW-TO] Installing and using RabbitMQ in Ubuntu

Posted on by
Reply

 

 

RabbitMQ is a powerful open source messaging system that implements the Advanced Message Queuing Protocol (AMQP) standard. Nowadays, there are thousand of applications that need a reliable messaging service to queue, for example, requests. Amazon offers a messaging service called Simple Queue Service (SQS) which is suitable for little project where the message load is not very heavy.

Amazon’s SQS could seem very cheap ($0.01 per 10,000 Amazon SQS Requests)  but if your application is repeatedly checking for new messages from several queues, the cost starts to matter. RabbitMQ is the solution. RabbitMQ is fast, free, supports clustering, easy-to-use, easy-to-install… However, there is one point that you need to take special care: availability. High-availability comes for free with Amazon SQS. You can find more information about RabbitMQ vs SQS here.

Let’s install and configure. We need to add a new repository to download RabbitMQ, so let’s open a new terminal and run:

sudo pico /etc/apt/sources.list

This file contains the repositories sources. Move to the end of the file and write:

deb http://www.rabbitmq.com/debian/ testing main

Now we need a public key to avoid warnings, so run:

wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
sudo apt-key add rabbitmq-signing-key-public.asc

It should have replied ‘OK’ when adding the key. So, let’s update and download RabbitMQ:

sudo apt-get update
sudo apt-get install rabbitmq-server

RabbitMQ comes with a handy tool called ‘rabbitmqctl’ to control the application (it must be used as root). To make our system a little more secure, we should set up a password for the ‘guest’ user. I am using ‘abcde’ as example. We can do this running:

sudo rabbitmqctl change_password guest abcde

Now everything is ready to start using it. Remember if you are using a remote server to open RabbitMQ’s default port 5672 in the firewall to accept connections. To test it, we are using the Official Java Client. You’ll need the libraries to run this code:

package com.hveiga;

import java.io.IOException;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.ConsumerCancelledException;
import com.rabbitmq.client.QueueingConsumer;
import com.rabbitmq.client.ShutdownSignalException;

public class TestRabbitMQ {

	private static String QUEUE_NAME = "TestQueue";
	private static ConnectionFactory factory = new ConnectionFactory();

	private static void init() {
	    factory.setHost("localhost"); // Host will be different if you are in remotes
	    factory.setUsername("guest");
	    factory.setPassword("abcde");
	}

	public static void send(String message) throws IOException {
	    Connection connection = factory.newConnection();
	    Channel channel = connection.createChannel();
	    channel.queueDeclare(QUEUE_NAME, false, false, false, null);

	    channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
	    System.out.println(" [S] Message Sent: '" + message + "'");
	    channel.close();
	    connection.close();
	}

	public static void receive() throws IOException, ShutdownSignalException, ConsumerCancelledException, InterruptedException {
	    Connection connection = factory.newConnection();
	    Channel channel = connection.createChannel();

	    channel.queueDeclare(QUEUE_NAME, false, false, false, null);

	    QueueingConsumer consumer = new QueueingConsumer(channel);
	    channel.basicConsume(QUEUE_NAME, true, consumer);

	    QueueingConsumer.Delivery delivery = consumer.nextDelivery();
	    String message = new String(delivery.getBody());
	    System.out.println(" [R] Message Received: '" + message + "'");
	}

	public static void main(String[] args) {
		init();

		try {
			send("HelloWorld!");
			Thread.sleep(2000);
			receive();
		} catch (Exception e) {
			e.printStackTrace();
		}

	}
}

 

More example and different uses can be found in the official site. In addition, Spring framework contains some method to interact with RabbitMQ. More information about Spring with RabbitMQ can be found here.

[HOW-TO] Auto-terminate an EC2 instance with Java

Posted on by
Reply

When we are doing different jobs in different instances, it is possible that we need to terminate an instance once the job is done. For this purpose we need to know the instance-id from within. Amazon provides a mechanism to get information about the instance itself: http://aws-musings.com/web-service-version-of-the-ec2-instance-metadata-query-tool/

With this mechanism, we just need to call a process from Java to get the instance-id and run an TerminateInstancesRequest. I have created a static Java method to terminate an EC2 instance:

public static void killMePlease() {

  try {
    String wget = "sudo wget -q -O - http://169.254.169.254/latest/meta-data/instance-id";
    Process p = Runtime.getRuntime().exec(wget);
    p.waitFor();
    BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
    instanceId = br.readLine();
  } catch(Exception e) {
    e.printStackTrace();
  }

  if(instanceId.equals("")) return;

  List<String> instancesToTerminate = new ArrayList<String>();
  instancesToTerminate.add(instanceId);
  TerminateInstancesRequest terminateRequest = new TerminateInstancesRequest();
  terminateRequest.setInstanceIds(instancesToTerminate);
  ec2.terminateInstances(terminateRequest);
}

I am not sure if there is other easier way to do it, but this code works. You can check the complete java code in my Github account: https://github.com/kec0/AWSKillInstance

Thank you!

[HOW-TO] Installing AWS SDK for PHP in Ubuntu 12.04

Posted on by
Reply

This little guide is meant to help people to install and set up the Amazon Web Services SDK for PHP in few steps. I’m going to try to explain every single detail to help new people working with Amazon Web Services.

To do so, I’m going to launch an Amazon EC2 instance with Ubuntu 12.04. To begin with we need to go the AWS Management Console, select the Amazon EC2 tab and click in ‘Launch instance’. We’ll follow ‘Classic Wizard’ and click Continue. Now we have to select an Ubuntu 12.04 (Precise Pangolin) instance. Since it is going to be difficult to find it among all AMIs, the easiest way is to check the AMI identifier in the Ubuntu website: http://uec-images.ubuntu.com/releases/precise/release/ . For this guide I’m going to select this AMI: ami-ac9943c5 (32-bit, ebs, us-east-1).

To find by AMI ID we click in ‘Community AMIs’ and we fill the Search field with the AMI ID we have selected. Something like this should appear:

Selecting Ubuntu AMI

We need to select the second one. In the next step we can select how many instances we want to launch and which type. For this example we will launch one t1.micro instance. Click continue. Click continue again. Give a name to the instance and click continue. Now you can select an existent keypair to connect to the instance or create a new one, it’s up to you. In the next window you have to select the security group for that instance (firewall). It is mandatory that the group allows SSH (port TCP 22) and HTTP (port TCP 80) connections at least, so if you need to create a new security group, do it with those rules. Click continue. Review all the parameters selected and if you are done click on Launch.

Perfect! Now we have an Ubuntu 12.04 running over AWS. So let’s start with the installation and configuration of the AWS SDK for PHP. We need to connect to our instance. We are going to do it using SSH (directly from terminal if you are on Mac or Ubuntu, or via Putty or any similar program if you are on Windows). We are not using the terminal provided by the AWS Management Console. To connect using SSH we need the public DNS of our instance. We can get this URL in the instances list in the Amazon EC2 tab, just selecting our instance and checking the information below.

Public DNS of instance

It should be similar to: ec2-107-21-164-34.compute-1.amazonaws.com. Let’s open a Terminal (or Putty) and browse to the folder where we have our keypair. Once there, we execute:

ssh -i keypair_name.pem username@public_dns

For my example, mine would be:

ssh -i myKeypair.pem ubuntu@ec2-107-21-164-34.compute-1.amazonaws.com

If everything is right, you will connect to the instance. However, not everything works at the first time.  If you cannot connect, the most common issues are:

  • Keypair permissions: the .pem file must have a specific permissions to be used. Those are that just the owner can read and write. So set them accordingly. If you are using Ubuntu or Mac execute: chmod 600 keypair.pem
  • Security group port closed: ensure that your security group allows SSH connections (port TCP 22).
Connected

Once you are connected, you are in a completely new and clear Ubuntu. So, we need to install the following:

  • Apache: it’s the webserver, needed to run PHP scripts.
  • PHP5: the engine of PHP.
  • Curl for PHP: a tool for transferring files that is used by the AWS SDK for PHP.
  • Pear: a PHP application repository.
  • AWSSDKforPHP: the SDK itself.

First of all, we should update the package list:

sudo apt-get update

And now let’s install what we need:

sudo apt-get install apache2
sudo apt-get install php5
sudo apt-get install php5-curl
sudo apt-get install php-pear
sudo /etc/init.d/apache2 restart

Everything, except the SDK, should be installed right now. Let’s test it. Browse to /var/www . Remove the file index.html (sudo rm index.html) and run sudo pico index.php . Pico or nano is a terminal text editor. Write ‘<?php phpinfo(); ?>’  and press Ctrl + X, then Y, and press Enter. Now if we go to http://ec2-107-21-164-34.compute-1.amazonaws.com (your public DNS address) in any browser we should see this:

PHP Installed

Nice! We are almost done. Let’s install and configure the SDK. We are going to do it using PEAR ( http://pear.amazonwebservices.com/ ), so let’s run:

sudo pear channel-discover pear.amazonwebservices.com
sudo pear install aws/sdk

Installation done. Let’s configure it. Navigate to /usr/share/php/AWSSDKforPHP . Once there, run sudo pico config-sample.inc.php . This file contains all the information to connect with Amazon when you are doing requests. You need to fill up key (that it’s also called public key) and secret key. You can get them from https://aws-portal.amazon.com/gp/aws/securityCredentials. When you are done, press Ctrl + X, press Y, and now rename the file to config.inc.php and press Enter and Y. Last step is to test if the SDK works. The SDK comes with some test files, so let’s use them! Without changing the directory, execute:

sudo cp -r _samples /var/www
sudo /etc/init.d/apache2 restart

Now navigate to /var/www/_samples/ and execute sudo pico cli-s3_get_urls_for_uplods.php and replace “require_once ‘../sdk.class.php’;” for “require_once ‘AWSSDKforPHP/sdk.class.php’;” . Finally, go to any browser and go to: http://ec2-107-21-164-34.compute-1.amazonaws.com/_samples/cli-s3_get_urls_for_uploads.php (using your public DNS address) and we should see:

Success Screen

Congratulations! You have installed successfully the AWS SDK for PHP in Ubuntu 12.04. Now you have uploaded 4 test files to a bucket in your S3 account, go check it! Each time you want to use the SDK you just need to add  ”require_once ‘AWSSDKforPHP/sdk.class.php’;” in your PHP code. I hope this guide helps people with installation and configuration problems. If you have any doubts, comment!

Héctor.