Passing Curiosity: Posts tagged cloud https://passingcuriosity.com/tags/cloud/cloud.xml Thomas Sutton me@thomas-sutton.id.au 2014-02-11T00:00:00Z Puppet Camp Sydney 2014 https://passingcuriosity.com/2014/puppet-camp-sydney/ 2014-02-11T00:00:00Z 2014-02-11T00:00:00Z Puppet Camps are regular, regional events for the Puppet community and this is the second or third time I’ve attended one. They can feel a tiny bit vendor-y (this should be unsurprising) but the quality of the talks and the attendees is pretty good, in my experience.

Nigel Kersten’s keynote talk was aimed at a pretty broad audience (a bit of Puppet, what’s driving uptake, etc.) but also described some of the new features in components included in the next release (IIRC) of Puppet Enterprise. I was particularly interested to learn about policy based auto-signing and trusted node data in Puppet 3.4+, external facts in Factor 1.7+, more readable ouput from Hiera 1.3+, and the news that Puppet Labs will be supporting some of their modules from the forge.

Peter Leschev from Atlassian described the process of introducing and developing “infrastructure as code” in the Atlassian build engineering team. He describe their introduction of a number of tools and measures and the impact on confidence in infrastructure changes being made. It was interesting to see the journey of adding code reviews, Puppet, Vagrant-based development (with Veewee), behaviour based testing (with Cucumber), continuous integration (Bamboo and Vagrant), profiling (Puppet’s --evaltrace flag), automated deployment (to staging) and notification (in HipChat). Later on I wished I’d asked if the graphs of confidence in his slides were from measurements, or for illustrative purposes only.

Lindsay Holmwood from Bulletproof described the Flapjack monitoring system – which seems pretty cool – and how you’ll be able to configure it with Puppet (when he releases the Puppet module). The architecture of Flapjack looked pretty interesting and I plan to have a play with it this weekend.

Rene Medellin spoke about NAB’s move to push some of their workloads into “the cloud” (AWS). They used Puppet as part of their SEO machine image building process and in deployment as one of their monitoring and compliance tools. Lots of Jenkins and automated building of AMIs and CloudFoundry templates and such.

Aaron Hicks from Landcare Research NZ spoke about the way he uses Puppet in a scientific research environment. Particularly interesting was the use of Puppet to formalise the configuration of the many, many precious snowflake machines used in the various research projects his organisation supports. The idea of supplying Puppet manifests to help in the replication of scientific computing sounds great.

James Dymond and John Painter from Sourced Group described a series of “Puppet in the AWS cloud” architectures they’d developed for clients in their consulting engagements. Most interesting was their fourth (I think) solution, where they implemented a “gateway” between AWS autoscaling notifications and Puppet, allowing the master to sign certificates, delete node reports, etc. as the AWS autoscaling system adds and removes nodes.

Matt Moor from Atlassian described the way they use Puppet to manage their SaaS offering. Each SaaS client has their own VM which, now, is managed using Puppet. This allows them to manage service and version dependencies much more reliably than their previous approach of building massive WAR files using Maven and managing them with hack-y shell scripts.

The last talk was by Chris Barker from Puppet Labs who gave a product demonstration of Puppet Enterprise. I’d already used most of the features demoed but some of the newer stuff – especially the event inspector – looked pretty cool.

Puppet Camp Sydney 2014 was a great event and brought to mind again just how much fun operations work (what little I’ve done) can be. In time, I expect the slides and videos of the presentations will be available from the Puppet Labs web-site on the Previous Puppet Camps page.

]]> PyconAU 2012: Implementing a cloud service in Python https://passingcuriosity.com/2012/cloud-services/ 2012-08-18T00:00:00Z 2012-08-18T00:00:00Z John Barham from SlickDNS.com.

A bit of blurb about “the cloud”.

Using

  • Using ECS and RDS for “main server”.

  • DNS servers on linode.

  • Postmark for email.

  • fabric

  • daemontools

  • celery on RabbitMQ. Decoupling form submission from processing to improve user experience; event logging (gathering metrics).

  • memcached for caching with Django’s built-in support; cache anonymous pages.

  • Browser caching: embed timestamps in the static media URLs, resources can be cached forever as the URL will change (http://gist.github.com/3375913).

  • API. Good old REST vs SOAP and JSON vs XML blah. HTTP authentication: basic over HTTPS.

  • Logging: CloudWatch to monitor EC2, is pretty heavyweight; StatHat for statistics and event logging.

See “The S Stands for Simple”

]]>