Github @ the Enterprise: Backups

Whenever you work on an enterprise and you have processes, quality specifications, and all that kind of stuff. One thing that will surely come up is backups. You must have backups of all your critical data. This is a good advice for everyone, not only enterprises.

So, what happens when you start working with third party providers, such as Google Apps, Github, Pivotal Tracker or others? They will probably provide you with SLAs, that include data recovery, backups, and other security consideration. Still you are never too careful. You never know when the next nuclear war is coming, when the internet is going down or whatever apocalypse scenery you want to imagine. So, we want to have backups near, reachable, in the case everything else fails.

We have recently started using github as our main git repository for our projects: private or public. We use  the git repository, and also Wiki and Issues services.

Getting a backup of a git repository it's easy: clone and fetch it once a day. And luckily, the wikis at github are also git repositories. Github issues is a little more complicated. You can access the github API and obtain all the issues, then you need to serialize and store all those issues. Luckily, git is a great tool to store versioned data.

In order to solve our problem, we have built a little gem to do the hard work. It creates a directory for each project and there it stores 3 git repositories: source code, wiki, issues. For issues, it creates a json representation of each.

You can find the source code at github and follow the instructions there. It's also available as a ruby gem so a: gem install backup-github would suffice.

We hope you'll find it useful!

Play! Framework Hackaton

Zauber team working

Zauber Software encourages employees to research new technologies in order to perform better and improve ourselves each day. So as to do this, we have Hackatons very often at Zauber in which we investigate different technologies .

In one of this Hackatons, we decided to test Play! Framework. The Play framework’s goal is to ease web applications development while sticking with Java. Its idea is pretty similar to Rails's idea.

We thought that the best way to test the framework would be creating an application that we'd use here at Zauber every day. In that moment, we started thinking about "A comerla!".

Everyday we call a lot of restaurants in order to have our meal delivered to the office. Right now, It's very complicated for us to know how much each meal costs, which is the total of the order, who's going to call the restaurant, who's going to collect the money to pay and when the order will be placed. "A Comerla" is the solution to this and the excuse to try Play!.

We also tried several Play! modules with this app. We used the recent developed Secure Social module for the Google Login. We also used Greenscript module minify the CSS and JS, to create dependencies between the JS files and to be able to use less files, among other things. In order to make the app more beautiful we used Twitter Bootstrap which makes it easy to style any web site. Furthermore, We heard that Heroku had native support for Play! Framework, so we decided to upload it there. You may see the app deployed at http://a-comerla.herokuapp.com/

We've open sourced the application so that anyone can access it. The url for the Github repository is https://github.com/zauberlabs/a-comerla. There, you'll find the instructions on how to deploy this in your own business. 

It was a really great experience and it's an awesome framework. We'll continue to test new technologies and post them here in the future.

2011 Tech Conferences Sum Up

In our eternal seek for knowledge, 2011 found us travelling to San Francisco, Portland and Sao Pablo to keep learning about technology. This is a late sum up of the conferences we attended.

JQueryCon (April - San Francisco)

A business trip turned into an educational one when we registered ourselves at the jquery conference. Two days of javascript, html5, jquery and all kind of frontend related stuff. Gabriel Baños and Mariano Cortesi went to this one. All the slides can be found here.
They are maybe a little old right now, but still, we recommend checking these talks:

• Mobile Performance by Steve Souders (slides)
• Progressive Enhancement 2.0: Because the Web isn't Print by Nicholas Sackas (slides)
• jQuery + Popcorn.js = Interactive, Immersive HTML5 Video Experiences by Rick Waldron (slides)
• Deferreds - Putting Laziness to Work by Dan Heberden (slides)
• Filling the HTML5 & CSS3 Gaps with Polyfills and Shims by Rey Bango (slides)

OSCon'11 (July - Portland)

Great technology conference organized by O'Reilly, with a focus on Open Source. Juan Codagnone, our open source advocate, attended. All decks & some presentation videos are already available here.

We recommend:

• What Would You Do With Your Own Google? by Steve Yegge (details and video)
• 4 Practical Uses for Domain Specific Languages by Neal Ford (details and slides)
• Java: The Good, Bad, and Ugly Parts by Josh Bloch. (details and video)
• The Expressiveness of Go by Rob Pike (details and slides)
• What Every Data Programmer Needs to Know About Disks by Ted Dziuba (details and slides)

QCon SP (September - Sao Pablo)

Second edition of qcon conference in Brazil. A Zauber crowd went to this event, our representatives were Juan Codagnone, Fernando Zunino, Christian Nardi and Marcelo Turrin. The conference was organized by our friends at Caelum. Most of the slides from the presentations are now available here.



We recommend:

• Performance Oriented Design by Rodrigo Albani de Campos (slides)
• Divida Tecnica by Alexandre Freire (slides)

LatAm Con on Agile Methodologies (October - Buenos Aires)

In our beloved city, an agile methodologies conference. Cecilia Hagge went there to represent Zauber. Lots of talks and workshops by local and international agile experts. Subjects as Story Writing, Estimation and Continuous Delivery where the stars. You can find a complete post on this conference on our blog.

RubyCon AR (November - Buenos Aires)

This one didn't require much travel, maybe a metro or bus ticket. Nevertheless, it was really interesting to make the first steps onto the ruby community in Argentina. Zauber has been for some time know mostly java centric, but this year we started learning new languages and platforms such as Scala, Groovy and, in this case Ruby. We hope to learn much more about it in 2012, we already have several ruby enthusiasts at the office. Juan Codagnone, Fernando Zunino and Juan Pablo Royo attended the conference.
We recommend:

• Optimizing for Happiness by Tom Preston-Werner (slides)
• Un Cuento de Tres Árboles by Scott Chacon (slides)
• Who Makes the Best Asado? by Aaron Patterson (slides)
• Monitoring with Syslog and Event Machine by Patrick Huesler (slides)
• Embrace NoSQL and Eventual Consistency with Ripple by Sean Cribbs (slides)

AnDevCon II (November - San Francisco)

Our first android conference. Mariano Cortesi and Andrés Moratti went to San Francisco to represent Argentina in this event. We really enjoyed getting to know the people in the android community. Some really interesting talks took place at the conference. The presentation decks are now available here.
Some talks we recommend:

• Deep Drive into Android Security by Aleksandar Gargenta (deck)
• Pragmatic Android Layouts by Kirill Grouchnikov (deck)
• RoboGuice by Michael Burton (deck1 , deck2)
• Tunning Android Applications (deck1, deck2)

QCon SF (November - San Franciso)

Final but not last, another big conference. Quite a classic in San Francisco, the QCon. A Conference organized by the guys at InfoQ, with a focus on architecture solutions, java, startups, and emerging technologies. Mariano Cortesi and Andrés Moratti represented Zauber at this conference. Videos for all the talks will appear an InfoQ along the next months, you can check the schedule here. If you can't wait, you can check the slides for all presentation at the qconsf site.




Some talks we recommend:

• Software Naturalism - Embracing the Real Behind the Ideal by Michael Feathers (deck)
• JVM Performance optimizations at Twitter's scale by Attila Szegedi (deck and video)
• Product Engineering by Mike Lee (deck)
• What the Heck is an API Platform Anyway? by Gregory Brail (deck)
• Building a Great Web API by Evan Cooke (deck)
• Dart in Depth by Gilad Bracha (deck)
• Objects, Anomalies, and Actors: The Next Revolution by Steve Vinosky (deck)
• Things I Wish I'd know by Rod Johnson (deck and video)
• Techniques for Scaling the Netflix API in the Cloud by Daniel Jacobson (deck)

That's all for 2011! We can add to the mix Barcamp, which we sponsored in BA, and then several people in Zauber went to Rosario and Parana to participate of the other Barcamps; also the Google Dev Day here en Buenos Aires.

Now 2012 has started and we are already planning which conferences we will attend! So if you have been to these conferences and want to add other talks that we missed, please comment!

Some thoughts on Agile Development

Last October we attended to the 4th Latin American Conference on Agile Methodologies

This event involved over more than 20 talks and workshops per day delivered by local, regional and international experts.

Here I focus on the main subjects treated and give you some tips you should not miss. Hope you find them interesting and useful!

Story Writing

Sessions in general were focused on determining a better product design figuring out the “why and who” is interested in a particular story.

Jeff Patton, who was in charge of the first keynote in the event, explained a technique used behind this concept: Story Mapping. Fortunately, and besides the amount of people who wanted to participate, we were able to attend to his workshop and gain some practical experience.

Jeff's vision about Scrum and Agile Methodologies in general is that they tend to enforce “output” during a project, i.e. deliver the biggest amount of functionality in a time-boxed period and finish stories already predicted in the backlog for that sprint. Nevertheless, he points out that's not the mean of a successful project. The goal is to satisfy the client (or the co-worker as he wants to call him/her) resolving the first needs that led him/her to actually desiring the product. The only way to accomplish this is by focusing on the project's ‘outcome’. In order to build the best product according to clients' expectations, it's necessary to work on a good release plan, understand the business and break the client role into the different stakeholders that will be taking part in the product.

Story Mapping consists in building a business flow, identify more detailed steps in the flow and establish a hierarchy between the more global steps which are going to become stories and their detailed tasks or sub-stories. Finally, the goal is to assign sub-stories to each release, mapping them with each stakeholder profile for whom the release is planned. Jeff emphasized on how important is the visual, self-organized and cooperative way of working with this technique, especially when human understanding is involved.

In my opinion, you should apply this technique in case you have the opportunity to decide on the product design. This happens in every software factory but outsourcing is not as far as you may think. If you have the chance, it's only about taking the initiative to participate and making the difference.

Regarding Zauber, it's good to confirm that we follow the right approach, getting involved together with our clients in product design and definition, acting not only as a development team, but also as product managers.

Estimation

There were some workshops introducing well known estimation techniques such as PERT applied effectively at enterprise level. Beyond the method itself, it is actually a good practice to think about 3 possible estimations for each story: the optimistic, the pessimistic and the realistic one. This is hard to incorporate but it would help a lot not only to achieve more precise estimations reducing the chance of sub and over estimation but also to identify the very true risks of the story itself.

On the other hand, there were discouraging talks about estimation. These theories are based on the new mixed methodology Scrum-ban which is the combination of both Scrum and Kanban. The new concept, following the story-points idea, is to estimate in stories' complexity or size and build a sprint according to the already known velocity team and the number of stories that can be delivered, without considering the usual hourly-men or ideal days unit. Even though this is a good approximation for reducing projects deviation and leading to a more comfortable work environment, I can't still figure out a way to apply it because clients are not yet prepared for this vision. People in general are used to work with time as their unit measure and it takes a long time to change from one paradigm to another. However, this is a nice theory to follow up and keep in mind.

Continuous Delivery

The talk exposed a truly typical problem concerning a complex and distributed architecture with frequently deploys. Features were easy to develop but integration tests and deployment consumed proportionally a lot more time leading to client’s upset and late delivery of functionality.

The team in charge decided to carry out an automatic system for test and deployment. It took them 1 month development and let them reduce their 10hs delivery time to some remarkable 20 minutes. During the talk, they explained the most important features to take into account when using continuous delivery and finally showed a demo.

The technology used for the project was: Jenkins + Plugin Promoted + Amazon. They casually mentioned they hadn't had good results with Cargo (another known plugin for Jenkins).

There was also an introduction to PAAS (Platform as a service), in particular they showed a little bit of Heroku and its main differences with Amazon.

Our advice is that, if you are about to develop or you're already maintaining a big project, you should adopt continuous delivery and choose the most suitable infrastructure between all the alternatives the market brings, taking into account your non-functional requirements.

I hope you've found this review interesting. If you want to know more about how we build software in an Agile way at Zauber, you can check Agile Software Development

.

Gnip4j: our open source Java library for Gnip

We are happy to announce the public release of Gnip4j, a new Open Source project developed at Zauber and released under the ASFL 2.0.

Gnip4j is a Java library to access Twitter premium feeds.

For more than a year, we've been developing real-time interactive data visualizations, based on social media, gathered mainly from Twitter. We have developed projects for the World Cup, the Brazilian Elections, to track the uprisings in Middle East and more. So far, we always used Twitter Streaming API with granted access to the Gardenhose.

But the requirements of new projects and our ambition to create even more complex and accurate visualizations and real-time dashboards, made us start considering new data feeds. That's why we are now working with Gnip4j and using their premium feeds to take advantage of Twitter firehose to process 100% of tweets.

Our Gnip4j library is the result of our work using Gnip services and we are happy to contribute it as Open Source software, looking forward that other developers can use it and contribute back.

You can get involved by following the instructions here.

Please give us your feedback and feel free to contribute back.