Posted:
November has gone by like this:

Okay, maybe not that fast, but let’s rewind a bit and see what we’ve gotten up to this month.

Google Cloud Platform Live introduces Container Engine, Cloud Networking, Cloud Debugger, and more
What do you get when you get ten of thousands of developers from around the world joining in for one event? A whirlwind. On November 4, Google Cloud Platform Live featured keynotes, highlights from our customers, sessions covering topics from mobile apps to cloud computing, and some fiery moments (literally). We also announced new features that are now available, including:

  • Google Container Engine
The alpha release of Google Container Engine lets you move from managing application components running on individual virtual machines to launching portable Docker containers that are scheduled into a managed compute cluster for you. Want to learn more? Check out this post.

    • Google Cloud Networking
    We’ve been investing in networking for over a decade at Google to ensure that you always get the best experience. With Google Cloud Platform, we’re focused on bringing our customers that same scale, performance and capability. So with Google Cloud Interconnect, it’s now easier for you to connect your network to us.

      • Firebase = building faster mobile apps
      During the event, we showed how Firebase helps you build mobile apps quickly and also gave a sneak preview of some new features the team has been hard at work on. Read more on what Firebase and Google Cloud Platform are getting up to together here.

        • Cloud Debugger
        Cloud Debugger is available in beta and makes it easier to troubleshoot applications in production. Now you can simply pick a line of code, set a watchpoint and the debugger will return local variables and a full stack trace from the next request that executes that line on any replica of your service. There is zero setup time, no complex configurations and no performance impact noticeable to your users. Ready to get started? Check out more info here and try it for yourself.

          To get a full recap of the event, check out our detailed blog post covering all our announcements. Or re-live the event with the keynote, session videos, or a live-tweet of the whole day.

          Curated Ubuntu Images now available on Google Cloud Platform
          Our customers spoke and we listened. Google Cloud Platform announced this month that we are now offering Ubuntu 14.04 LTS, 12.04 LTS and 14.10 guest images in beta - at no additional charge. We’d love to have you take these images for a trial run. Try it here and let us know what you think.

          Compute Engine welcomes Autoscaling and Click to Deploy Percona XtraDB Cluster
          This month we introduced two highly anticipated features to Compute Engine: Autoscaling and Percona XtraDB Clusters. Autoscaling allows customers to build more cost effective and resilient applications. Click to Deploy Percona XtraDB Cluster helps you simply launch a cluster preconfigured and ready-to-use in just a few clicks.

          Free IPv6 address with Cloud SQL instances
          It’s no secret that the Internet is quickly running out of IPv4 address space. Google has been at the forefront of IPv6 adoption -- the newer, larger address space -- and we are now assigning an immutable IPv6 address to each and every Cloud SQL instance, and making these addresses available for free. Full details can be found in this post.

          AffiniTech and Framestore: Saving money and creating great customer experiences through Google Cloud Platform
          We heard first hand from our customers AffiniTech and Framestore on how they use our products. AffiniTech helps their customers make data-driven decisions, while Framestore creates unforgettable visual effects (think Gravity). Both companies have seen incredible impacts to their bottom line and with their customers. But don’t just take our word-- hear it directly from AffiniTech and Framestore in their blog posts.

          November, and this year, has given us a lot to be thankful for, so we want to say thank you to all of you who’ve helped us hack, hustle, and go on this journey. To all those outside of the US, we wish you a well-deserved thank you for your support and to those stateside, have a Happy Thanksgiving.

          -Posted by Charlene Lee, Product Marketing Manager

          Posted:
          Implementing high-availability for MySQL can easily become complicated with so many options, including some that go a step further in combining high-availability with multi-master replication in a cluster format and more. Not to mention that besides MySQL’s own replication functionality, separate open source projects also include Tungsten Replicator and Galera, amongst others. It’s clear that each option has pros and cons, and selecting the best option for your specific use case can require a lot of research and testing.

          That's why today we’re going to try making this process easier by introducing Click to Deploy Percona XtraDB Cluster to help you simply launch a cluster preconfigured and ready-to-use in just a few clicks. Percona is an open source company supporting MySQL that provides a convenient package called Percona XtraDB Cluster, which combines Percona Server with Galera replication software. Using Percona XtraDB Cluster, developers can set up and configure a multi-master MySQL cluster with additional performance, scalability and diagnostic improvements over standard MySQL. Plus, Percona software is open source and compatible with existing MySQL environments.

          Along with Percona XtraDB Cluster, each server node includes Percona Toolkit, a set of command-line tools for administering MySQL, giving developers more flexibility in their options for MySQL clustering. Our goal with this Click to Deploy application is to provide an environment to evaluate Percona XtraDB Cluster as a solid option in high-availability MySQL. We want to help illustrate how Google Compute Engine can be used on a team already using MySQL, so you can build your applications however way you like.

          Learn more about running Percona on Google Compute Engine and deploy a Percona MySQL cluster today with our free trial credit. Please let us know what you think about this feature and the challenges with scaling MySQL in the cloud. You can also contact Percona for Consulting, Support, Managed Services, and Training. Deploy away!

          -Posted by Chris Pomeroy, Program Manager

          Posted:
          Today’s guest blog comes from Steve MacPherson, chief technology officer for Framestore, a visual effects production firm headquartered in London. Framestore’s visual effects work has been seen in films like “Avatar,” “Gravity,” and “Guardians of the Galaxy,” winning the company numerous Academy Awards, BAFTAs, and Cannes Lions awards.

          If you saw “Gravity,” hopefully you enjoyed the movie’s depiction of space travel and weightlessness. It was a brilliant experience to take on such a daunting project, and very rewarding to be part of the collective effort.
          A lot of planning and effort goes into every visual effect, as well as a fairly large amount of computing power. At peak times – when we’re rendering images on several projects at once for advertisers and film studios – we’ll consume the processing power of up to 15,000 Intel cores. Managing peak provisioning and matching resources to projects is central to keeping the production pipeline moving toward various deliveries. The challenge that returns regularly is when demand for resources conflicts with capacity – usually during periods when the stress of delivery is at its highest, and the focus is on realizing the creative goals of our clients in time for the immovable object that is a major release date.

          Historically, this boiled down to a simple scenario: purchase additional equipment. A design maxim I've long held is that we've never built a machine room that doesn't eventually run out of space, cooling or power. This “computational load bubble” is a result of both the scale of modern studio films, and the fact that we run multiple films through the facility in parallel. This is the peak provisioning problem that we were looking to address for a number of years. Once films are delivered, the demands recede, and we have an excess capacity for some period of time until we reach the next set of deadlines.

          For the past few years, we’ve kept a close eye on the potential for using external resources as an overflow valve. Google, through its Google Compute Engine, is the first company we've worked with that was able to combine raw resources with a team that understood our requirements in detail and a business model that helped us manage the economics. Within a day of firing up the network, we built our image inside the Compute Engine container.

          At Framestore, we’ve developed a sophisticated in-house job submission system based on our render queue manager, fQ – it’s extremely efficient at juggling various job types to match them accurately to rendering nodes available at any given time. This workflow is central to the sustained high levels of efficiency for our render farm, giving us up to 95% of overall capacity for weeks on end.

          Having Google Compute Engine on the back end opened a number of opportunities for us to siphon off a certain class of work during a period of peak production. The load reduction on our farm allowed us to be much more specific in how we prioritized our deliverables, ultimately leading to a much more focused and predictable delivery schedule – great for production and great for maintaining confidence with the studios.

          Google gives us breathing room during periods of peak capacity, allowing our artists more flexibility around creative refinement. We do many iterations of an image or a visual effect, making minor technical tweaks and submitting it to the render farm to see if it works. During periods of high load, if all of our rendering is in-house, the creative team might have to wait more than a day to see results when the in-house farm is at capacity. This introduces stress and management overhead around which shots get priority.

          By adding Compute Engine to our workflow and allowing our in-house capacity to focus on the studio work, everyone’s project gets computing time – and the creative team can get as imaginative as they want to, with fast views of new iterations.

          The results: fewer bottlenecks, more creativity and more predictability, not to mention saving about £200,000 (more than $300,000 USD) on the cores we didn’t need to buy. We can now confidently move into final stages of production on our biggest projects, knowing we have a reserve of computational ability on tap. When you check out new movies this year and next like “Dracula Untold” and “Jupiter Ascending,” you’ll be looking at our visual effects work, created using all the computing power at our fingertips.

          Posted:
          The true power of cloud computing is unlocked when developers can build resilient and cost efficient applications that use just the right amount of resources necessary at any given time. So the same team that designed the scaling infrastructure for products like Google Search and Gmail have brought a highly anticipated feature to Google Compute Engine - intelligent horizontal Autoscaling. Today we are releasing the service into Beta, which means it is now available for everyone to start using.

          Autoscaling allows customers to build more cost effective and resilient applications. Using Compute Engine Autoscaling, you can ensure that exactly the right number of Compute Engine instances are available at any given time to handle your application’s workload. This saves you money when your application’s usage is low, and ensures your application is responsive when utilization is high.

          The Compute Engine Autoscaler is able to intelligently and dynamically scale the number of instances in response to different load conditions by defining the ideal utilization level of a group of Compute Engine instances. This means that when the actual utilization of your service increases or decreases, Autoscaler will detect the change and adjust the number of running instances to match. Autoscaler can respond to a number of different metrics such as CPU load, QPS on a HTTP Load Balancer and metrics defined using the Cloud Monitoring service.

          One early customer of Compute Engine’s Autoscaler was Wix.com, the popular website-building service. Golan Parashi, Wix.com's Infrastructure Team Lead, commented how Google uses heuristics to determine how many instances to add at one time to hit demand, “reducing [our] expenses, while giving us confidence that Google will manage the appropriate number of machines, even when a spike occurs."

          Autoscaler not only chooses the right number of instances but also adapts automatically based on how far the current state is from the desired target. This means Autoscaler performs well even in unexpected scenarios such as sudden traffic spikes. At Google Cloud Platform Live, we demonstrated how an application could scale from zero to handling over 1.5 million requests per second using Autoscaler.

          Here are some additional resources to get you up to speed on Compute Engine’s Autoscaler:

          We can’t wait to see what you build - and scale - next on our platform.

          -Posted by Filip Balejko, Software Engineer

          Posted:
          Today’s guest blog comes from Vincent Heuschling, founder and CEO of Affini-Tech, a creator of data platforms that help businesses make data-driven decisions. Affini-Tech is based in Meudon, France and was founded in 2003.

          Affini-Tech’s primary goal is helping our customers make data-driven decisions, regardless of the industry they work in. This means giving them easy workflows and web-based interfaces for analyzing and managing Big Data – all at a cost that makes sense for their businesses. Google Cloud Platform has become the foundation for everything we do.

          When we launched Affini-Tech, we knew we needed a scalable solution for building applications and analyzing information. We explored a number of Cloud vendors - including doing an initial storage deployment on another large public cloud. However, after trying Cloud Platform and speaking with the Cloud Platform team, we decided to go all-in with Google. Google's pricing was the most competitive, but we also found it to be the best platform for our developers. Google App Engine, Google Compute Engine and Google BigQuery provided us with an integrated technology stack that worked better than anything else on the market, making it easier for us to build complex applications.

          We use App Engine to build applications that help our customers to control their data collections, model data sets and filter and group data. We sell these applications to marketing companies that want to run their software on top of our platform. App Engine is flexible enough to allow developers at marketing companies to customize our stack for their own needs.



          Cloud Platform also helps us generate data findings at a faster rate. We’re storing data in Google Cloud Storage, creating ephemeral Hadoop and Apache Spark clusters, then pushing the data into BigQuery for analysis. Ephemeral clusters provide a more efficient, flexible and cost-effective processing model than old-fashioned static clusters and take full advantage of the Cloud model of computing. The key enablers to using the ephemeral clusters are the Google Cloud Storage connector, which lets us directly access data on Cloud Storage using standard Hadoop interfaces, and bdutil, which helps us automate cluster deployment. Our customers only have to “pay as they process,” which saves them money. Not to mention, setup takes less time. Traditional clusters can take days to install, whereas we can get ephemeral clusters up and running in just minutes.

          We can pass these cost savings onto our customers, which makes our products and services more competitive. Many of our users are used to spending more than $250,000 to build a data analytics platform. We can often provide the same service for $2,000 per month. This saves our customers money and allows them to go deeper with their data analytics. This access allows our customers to create things like micro segments in their customer base so they can do better targeting for their marketing campaigns.

          In a way, Google is helping to democratize data, since more businesses can afford to study it. If a customer is already using Google Apps – and many of them are – we can integrate our data platforms into Google Apps, making these tools even easier to use and understand.

          As a small company, the support we receive from the Cloud Platform team is helping us think bigger. It enables us to build new tools and platforms that take advantage of big data. We plan to make a push for business beyond France and the retail sector – and we’re confident about our expansion, with Cloud Platform doing the heavy lifting.

          - Contributed by Vincent Heuschling, founder and CEO of Affini-Tech

          Posted:
          The Internet is quickly running out of IPv4 addresses (the traditional IP addresses, e.g. 173.194.251.158), which has led to rationing strategies such as having to pay for reserved addresses. The industry response to this issue has been to develop a new, much larger, address space: IPv6 offers an abundance of available IPs (of the form 2001:4860:4864:1:329a:211f:1d19:a258). Google has been at the forefront of IPv6 adoption, and we are now assigning an immutable IPv6 address to each and every Cloud SQL instance and making these addresses available for free.

          Everything is ready for you to benefit from IPv6: you can see the IPv6 address of your instance using the Cloud SDK command-line tool. With the current version of the tool, the command is:

          gcloud sql instances describe

          The response will be of this form (we’ve bolded the IPv6 address and shortened the output for readability).

          currentDiskSize: 82.4 MB
          databaseVersion: MYSQL_5_5
          instance: testinstance1
          instanceType: CLOUDSQL_INSTANCE
          ipAddresses:
          [
          ipAddress: 173.194.251.158
          ]
          ipv6Address: 2001:4860:4864:1:329a:211f:1d19:a258
          maxDiskSize: 250.0 GB
          (...)


          Alternatively, you can see it in the Google Developers Console (in the Properties section on the instance overview page):

          All new and existing Cloud SQL, instances will be assigned an immutable IPv6 address. Please note however that in order to use it you need to explicitly authorize your Cloud SQL instances to receive IPv6 traffic. You can do this using the same access control mechanism you use to control IPv4 traffic, by explicitly authorizing external IPv6 addresses that are allowed to connect to your instance. For details, see how you can configuring access control for IP connections.

          If you currently connect to your instance over IPv4, you can continue to do so, or you can switch to connect over IPv6. If you make the change, you can then choose to release your IPv4 address and stop paying for it. To do so, follow the instructions to edit the properties of a Cloud SQL instance.

          We understand how frustrating it is to optimize for artificial constraints, like a limited address space, which is why we’re bringing IPv6 to Google Cloud SQL so you can focus on your applications instead. For any question please join us on Stack Overflow using the google-cloud-sql tag.

          -- Posted by Easwar Swaminathan, Software Engineer

          Posted:
          Last week at Google Cloud Platform Live, we demonstrated how Firebase could be used to build mobile apps quickly and also gave a sneak preview of some new features the team has been hard at work on.

          One highlight onstage was showing how you could build a fully functional collaborative office planning application in about 15 minutes. Building this app so quickly was possible because Firebase provided the common backend infrastructure needed to get up and running, allowing us to focus on the application’s front-end code and user experience.

          We’ve been busy
          We’re proud of what Firebase already can do to speed up your development time, but we’re working harder than ever to improve the platform. At last week's event, we announced a new Firebase feature called enhanced query support. You can now query data by any child key and have that query update in realtime as your data changes. These improvements make it easier for developers to sort and filter their data and will make many common use cases much easier to implement.

          Last week, we also demoed a new type of integration with server-side code that makes it especially easy to connect Firebase with Google App Engine. This new feature, called Triggers, is simple to implement: in your Firebase’s rules, you define a Trigger specification which includes criteria for when the trigger should fire. When the criteria are met, an outbound HTTP request is sent to an external service you specify along with any data and headers you provide. Use triggers to submit a payment, send an SMS, ping a server for logging and analytics or invoke some code on an external server -- all without needing to write your own server-side code.

          We used Triggers to connect the office planning application to an analytics backend running on App Engine, and then used BigQuery to generate a report about how the furniture moved around the room -- all in just a few minutes with a couple lines of code. Our Triggers feature hasn’t launched yet, so stay tuned for our beta announcement soon.

          And are going to get even busier...
          We’re continuing to improve Firebase along every dimension, and a big part of that effort will go towards improved integration with the rest of Google Cloud Platform. Although developers building Firebase applications love the simplicity of focusing on client-side development, about half of our developers also run their own server-side code. These developers need to do computationally intensive tasks like video or image processing, perform complex analysis on their data, and keep proprietary business logic on trusted servers. With Firebase joining Google Cloud Platform, developers can power their entire product using a single platform.

          This is just the beginning. We’re focused on finding all of the ways Firebase can work seamlessly with other Google Cloud Platform products to bring you the best developer experience possible. To get these updates first, please follow us on Twitter.

          As always, we look forward to seeing what you build.

          -Posted by Andrew Lee, Product Manager