Presentación en Barcelona GTUG

Paso a detallar los puntos que sugiero presentar para el GTUG de Barcelona en la próxima reunion. Cualquier sugerencia es bienvenida, así que si tienen alguna, solo basta que me envíen un email o que comenten en el blog.

Título: Implementando RESTful en GAE
Cuando: Miércoles 20 de Junio 2012
Hora: 18:15h
Lugar:
FIB - UPC Campus Nord - Sala de actos B6
Calle Jordi Girona, 31, 08034 Barcelona, Spain
08034

Temario

• Repaso a configurar un proyecto Java en GAE
• Uso de persistencia con JDO
• Gestion de las instancias y servicios CRON
• Configuración del plugin de maven para GAE y JAVA
• Libreria RESTlet para Android y GAE
• Ejemplos de implementacion en un sitio web real
• Discutir patrones de interacción/sincronización con servicios RESTful
• Manejo de Errores en la capa de comunicación 
• Uso de threads para mejor respuesta
• Implementación RESTful en Android tomando en cuenta el ciclo de vida de una actividad

Controlling your Task Queues

Summary

One of the key features in any framework for executing tasks (serializable and persistents) is how you can control them when things go wrong. This posting discusses some of the control panel of the GAE task queue. Details of the process is described in another article.

Previous Work

On my previous entry I wrote about the plans of creating automated tasks to process 130,000 transactions.Today I write about how things did not go as expected.

Executing the Tasks

Everything went out of control and very quickly. There were some important limits that I needed to respect in the Google Maps API. The most important wass the 120,000 transactions a day limit. With that I had no trouble. However, there must be a limit of transactions per minute that is nowhere published. With the limits in the GAE tasks I had no trouble.

However, I realized that after processing around 800 transactions, the google Map API was giving me errors. When I looked at the log, I realized I had fullfilled the quota for the day. I initially expected the quota to be 2000 transactions a day. But whenever I started executing the tasks, I set the limit to 5 transactions a second. I quickly reached a total of 300 transactions a minute, thus fulfilling my quota.

Since I promised to my wife that I would have all the data processed by Thrusday, I started fearing that I had to sleep in the couch. The previous day I was bragging about all the computer power that I had available and today i had reached the maximum.

Controlling the Tasks

The console for the tasks in GAE is your best friend in this circumstances. Also make sure that during the different loads generated that you follow the best practices for loads.

The tasks api actually worked as expected but the Google Maps API was the first thing to break down. I had anticipated a processing of 200

Integration as a Service in the Cloud

It is no surprise that the new wave of system integration tools are being designed with the cloud in mind. What really surprises me is the easiness that allows a very heterogeneous environment seem completely simple to the developers that use it. This article discusses the latest features of Apache SMX Fuse (http://fusesource.com/) , a systems integration platform for the cloud.

For those who haven’t tried Apache SMX Fuse, it is an excellent integration solution. It is based on several Apache projects: CXF, HttpClient, Camel, Zookeeper, among others. And allows the integration of almost any type of system: via ftp, file, web service (SOAP and RESTful), email, plain text file, etc. We have use it for several projects and is a very powerful tool.

Going back to the IaaS for the cloud, one of the most difficult things in configuring high reliable systems (clusters, pools, master/slaves, you name it) is the configuration management (CM) -- I am following here the SE Book from T-Systems. Some solutions like Oracle Weblogic are somehow better than others but at the end, when you are creating containers, app modules, shared libraries, etc it is very difficult to be error free. That’s why I was glad to see that the folks at SMX Fuse have applied some very good patterns:

Inheritance
One of the pilars of Object Oriented Design, inheritance allows you to provide a very efficient configuration of modules. Based on Zookeeper (http://zookeeper.apache.org/), the Apache project, it helps you to be consistent and achieve excellent repeatability.

Provide Different Sizes: Small, Medium, Large
Why order a large pizza when you only need a slice? That’s definitely a problem when dealing with these high replication systems: they are usually really heavy and complex. Apache SMX Fuse now comes with different sizes that allows you to use just what you need.

Bet on Auto-Discovery
Yeah, we all have problems with the famous JNDI registry, but it is an start. I have seen some nice features of this integration platform that is really worth looking. An excellent fallout system that allows the delivery of messages through different routes: JMS, WebServices, etc. and do not even worrying if the system is on-premise or on-public.

The bottom line of CM for the cloud is to have zero downtime for the developers, reduce all the proliferation of dirty system (VMWare images everywhere ), and ease the rollout.

preua

hola

Cloud Enhancement for your Smartphone

A couple of weeks ago I decided to develop an app intended to replace our grocery list at home -- we normally keep all items in a sheet of paper. The first alternative was to do the web app in HTML5 since everybody is talking about the power of the new javascript libraries, etc. so i couldn’t wait to implement it.

For the HTML5 version I developed a website (soon to be revealed to the general public) that allows users to keep grocery items under control. The website turned out to be more advanced than what I was expecting. It has plenty of business logic. There were items that if labeled weekly, would require a reschedule. There were other items that were “Reminder Only” to be scheduled some day in the future. And there were some other items that required a description in case there was a sale. Yes, my wife and I went crazy with the design. Therefore, I kept all the rules in the business layer and worked on the presentation layer leveraging on html5 to reduce the development time. (I used GAE for the cloud).

The first HTML5 release was a total failure. Although I did my testing, I did not realized how bad the 3G network coverage inside the commercial center is. Each time I marked an item as “checked out” it took forever for the connection to respond. I was always lagging a couple of aisles behind my wife trying to keep up with what had already been bought and what was pending. No wonder why I marked that we had bought soy milk when in fact we did not! (We just bought milk).

So it was time to go RESTful and do some more cloud enhancements. For the new redesign, most of the data is kept local vs 100% online. At the time the Android App is initiated, the app downloads the lists and the items for the specific user. It then saves them in the local database. The user then edits the items locally and syncs them at will. This design proved to be better than the HTML5 version. It was faster and more responsive.

Cloud Experience
It was pretty easy to set up the RESTful services in GAE. Although they do not provide a framework for those services, the open source community is pretty active on this area. I used Restlet, a framework with a pretty light library and pretty active user community, fresh documentation, and easy to use API. And because my friends always complain that I use Python, this time I used Java for GAE. The Restlet Framework has customized a release for Android apps, too. I also noticed that datanucleus, the persistence framework behind the Java GAE, also had RESTful interface but I did not use it.

There are some factors to consider while exposing your RESTful services using the cloud. You have to think about the warm-up period. Due to the nature of the cloud, most providers can put your instance to sleep if there is no activity. When the instance is brought back to life, your end user looses some precious seconds loading content. So I activated the warm-up functions in GAE and also updated some of the cron definitions for activating my persistence manager. After that the response time was down from 10 seconds to 600 ms. Huge improvements.

Other Frameworks
I was amazed at how many frameworks are there for Android and iOS cloud offerings. There are companies that give you an SDK and basically you can store all your data in their cloud for a fee. They take care of caching, serialization, security, geolocation, and performance. Funny because those are the same services that you already get from GAE or Azure. Some of the companies that I checked were Parse , Cocoafish , Urban Airship ,and Kii Cloud to name a few.
Conclusions
There is definitely an advantage in going cloud for Smartphone apps. One of the key factors for the clod storage is that most datastores are NoSQL. Therefore, your schemas can be modified without the pain of foreign keys or any restrictions which is great for the release cycle of Smartphone Apps that usually is every two weeks. So it is a very good solution.

Android development

These days I'm mostly developing Android apps. We'll see how it goes.

Effective Management: Which are the most valued traits in my boss

Recently, I read an article about Google's efforts inside to determine what makes a good manager. I was struck to find out that technical expertise did not rank among the top attributes of a good manager. Given my technical background, this remark sounded counter intuitive. Reflecting upon the article and remembering my former bosses, I changed my mind and agreed with it.

The most successful managers I had were not the techiest but instead the ones who: (1) gained an excellent understanding by just asking a few good questions, (2) those who related to the team, (3) and supported the professional development of the people they had in charge.

At least those are the main traits for me. Let me hear what do you have to say