Posted:
A new, free Udacity online course, Developing Scalable Apps with Google App Engine, helps Java developers learn how to build scalable App Engine applications. As you work through the course, you'll build a conference management application that lets users create and query conferences.

magnus pizza boxes 5-cropped.png

The course starts with an entertaining introduction to Platform as a Service(PaaS). Magnus Hyttsten, Google Developer Advocate, discusses the evolution of server-side computing, from apps that could run on a computer under your desk, to applications that require the computing power of a data center. (This is not without disadvantages, as he points out. "It is no longer possible to warm your feet on the fan outlet.")

Urs Hölzle, Senior VP of infrastructure at Google, gives the background on Google Cloud Platform: "We built our internal cloud a long time ago, for our own needs. We had very large-scale applications, so we needed a very capable cloud, and now we're making our cloud available to everyone. In Cloud platform, App Engine is the one system that makes it really easy for you to start very small and then scale to a very large user base."

just the sandwich 3.png

After learning about the evolution of the data center and the need for scalability, you'll get right down to business and learn how to store data in the Datastore, use Memcache to speed up responses and cut down on Datastore quota usage, write queries, understand indexes, and use queues for tasks that execute outside front end requests.

Along the way, you'll build a backend App Engine application that uses Google Cloud Endpoints to expose its API to other applications.


conf-central-code.png

You'll learn how to implement Endpoints to make the API available externally, and how to use the Endpoints API from an Android application.

If you take this course, you'll not only learn about App Engine, but you'll use it behind the scenes too. Udacity uses App Engine to serve its online courses. Mike Sokolsky, Udacity co-founder and CTO, talks about Udacity's decision to use App Engine to host Udacity's MOOCs. He says, "It pushes you in the right direction. It pushes you to the best design practices for building a scalable application." And that's what this course aims to help you do, too.

You can take the course, Developing Scalable Applications with App Engine, at www.udacity.com/course/ud859.

The full course materials — all the videos, quizzes, and forums — are available for free for all students by selecting “View Courseware”. Personalized ongoing feedback and guidance from coaches is also available to anyone who chooses to enroll in Udacity’s guided program.

For more courses that Google and Udacity are developing together, see www.udacity.com/google.

"
-Posted by Jocelyn Becker, Developer Advocate

Posted:
Starting today, Google Cloud Monitoring Read API is generally available, allowing you to programmatically access metric data from your running services, such as CPU usage or disk IO. For example, you can use Cloud Monitoring Read API with Nagios to plug in to your existing alerting/event framework, or use it with Graphite to combine the data with your existing graphs. Third party providers can also use the API to integrate Google Cloud Platform metrics into their own monitoring services.

Cloud Monitoring Read API allows you to query current and historical metric data for up to the past 30 days. Also, you can use labels to filter data to more specific metrics (e.g. zones). Currently Cloud Monitoring Read API supports reading metric time series data from the following Cloud Platform services:
  • Google Compute Engine - 13 metrics
  • Google Cloud SQL - 12 metrics
  • Google Cloud Pub/Sub - 14 metrics

Our documentation provides a full list of supported metrics. Over time we will be adding support for more Cloud Platform services metrics and enhancing the metrics for existing services. You can see an example of usage and try these metrics for yourself on our getting started page. For samples and libraries, click here.

Example: getting CPU usage time series data
GET \
https://www.googleapis.com/cloudmonitoring/v2beta1/ \  # Access API
projects/YOUR_PROJECT_NAME/ \                          # For YOUR_PROJECT_NAME
timeseries/ \                                          # get time series of points
compute.googleapis.com%2Finstance%2Fcpu%2Fusage_time?\ # of CPU usage
youngest=2014-07-11T10%3A29%3A53.108Z& \           # with this latest timestamp
key={YOUR_API_KEY}                                     # using this API key
Your feedback is important!
We look forward to receiving feedback and suggestions at cloud-monitoring-feedback@googlegroups.com.

-Posted by Amir Hermelin, Product Manager

Posted:
Today’s guest blog comes from Rafael Sanches, engineer at Allthecooks, a social media platform for people who love cooking. Allthecooks is available on Android, iPhone, Windows Phone, Google Glass and Android Wear, and is a top recipe app on Google Play.

At Allthecooks, we’re connecting passionate chefs with casual and first-time cooks on every major mobile device including Android phones, tablets, watches and Google Glass. People use our app to find, rate and comment on dishes they can cook themselves, or post their own ideas complete with directions, ingredients, servings and nutrition information. We have chefs with tens of thousands of followers on Allthecooks, meaning that whenever they make an update we have to process hundreds of thousands of simultaneous API requests to feed that information to timelines of all their followers.

Creating a successful platform isn’t just about speed, it’s about scalability, too. Google Cloud Platform played a key role in helping us grow without worrying about our architecture. We launched in December 2012 with just three part-time engineers and have never taken funding, so building our own infrastructure was out of the question. Since launching, we’ve grown to over 12 million users with a million monthly active users. Our application now sees millions of interactions daily that run through Google App Engine and Google Cloud Datastore.

As our user base has grown, we’ve begun migrating the biggest pieces of our backend processing architecture from App Engine onto Google Compute Engine. This will allow us to operate at even higher performance levels by using the cheaper CPU and caching more data in the large RAM configurations supported by Compute Engine instances. We also plan to use Google BigQuery soon to make the process of finding the perfect recipe even easier.

Cost was a big concern when we were burning through our savings to build Allthecooks, but it wasn’t as important as making everything run as fast as possible. When a request hits, the response needs to be immediate. That’s why we built our recommendation engine on Compute Engine -- that lets us run entirely on RAM, so users get the results they need as soon as they need them. When you’ve got 60 seconds to sear a fish just right, you can’t be caught waiting on a laggy server request to see the next step in a recipe. We never want our app latency to stand between our users and a great meal.

There are plenty of cooking apps out there, but none with the same level of social interaction as Allthecooks, which is one of the biggest reasons our users spend so much time on the app. We let users ask questions and receive answers on every recipe, and upload pictures of their own dishes. Our recommendation engine in particular plays a pivotal role in making the Allthecooks experience so useful on mobile devices. Google Glass users love that they can find and follow a recipe instructions hands-free or tilt their heads up to see the ingredient list. They can even record their own recipes using voice, photos and video and send it to their Allthecooks account. All of this requires a reliable infrastructure.

We don’t have a system administrator, so we need a system that’s reliable, manageable and scalable while requiring minimal oversight. Cloud Platform gives us confidence that the app is stable, and it can even be left alone for a week without requiring maintenance. Google’s support team has also given us peace of mind. They’re always quick to respond, and they have provided great services the few times we needed help.

It gives us great pride that Allthecooks has helped helped millions of people live healthier, and discover new foods and products they love. We believe in launching early and listening to what our customers want. Cloud Platform is a crucial component to our success to date and strategic for our future growth.

-Posted by Rafael Sanches, co-founder and engineer of Allthecooks

Posted:
If you saw our post about Cassandra hitting 1 million writes per second on Google Compute Engine, then you know we’re getting serious about open source NoSQL. We’re making it easier to run the software you love at the scale you need with the reliability of Google Compute Platform. With over a dozen different virtual machine types, and the great price for performance of persistent disks, we think Google Compute Engine is a fantastic place for Apache Cassandra.

Today, we’re making it even easier to launch a dedicated Apache Cassandra cluster on Google Compute Engine. All it takes is one click after some basic information such as the size of the cluster. In a matter of minutes, you get a complete Cassandra cluster deployed and configured.

Each node is automatically configured for the cloud including:
  • Configured with the GoogleCloudSnitch for Google Cloud Platform awareness
  • Writes tuned for Google Persistent Disk
  • JVM tuned to perform on Google Compute Engine instances

The complete set of tuning parameters can be found on the Click to Deploy help page.

So, get out and click to deploy your Cassandra cluster today!

Learn more about running Apache Cassandra on Google Compute Engine at https://developers.google.com/cloud/cassandra.

-Posted by Brian Lynch, Solutions Architect

Cassandra, is registered trademarks of Apache, Inc. All other trademarks cited here are the property of their respective owners.

Posted:
We’ve had a great time giving you our predictions for the World Cup (check out our post before the quarter-finals and semi-finals). So far, we’ve gotten 13 of 14 games correct. But this isn't about us picking winners in World Cup soccer - it’s about what you can do with Google Cloud Platform. Now, we are open-sourcing our prediction model and packaging it up so you can do your own analysis and predictions.

We used Google Cloud Dataflow to ingest raw, touch-by-touch gameplay day from Opta for thousands of soccer matches. This data goes back to the 2006 World Cup, three years of English Barclays Premier League, two seasons of Spanish La Liga, and two seasons of U.S. MLS. We then polished the raw data into predictive statistics using Google BigQuery.

You can see BigQuery engineer Jordan Tigani (+JordanTigani) and developer advocate Felipe Hoffa (@felipehoffa) talk about how we did it in this video from Google I/O.

Our prediction for the final
It’s a narrow call, but Germany has the edge: our model gives them a 55% chance of defeating Argentina due to a number of factors. Thus far in the tournament, they’ve had better passing in the attacking half of their field, a higher number of shots (64 vs. 61) and a higher number of goals scored (17 vs. 8).

But, 55% is only a small edge. And, although we've been trumpeting our 13 of 14 record, picking winners isn't exactly the same as predicting outcomes. If you'd asked us which scenario was more likely, a 7 to 1 win for Germany against Brazil or a 0 to 1 defeat of Germany by Brazil, we wouldn't have gotten that one quite right.

(Oh, and we think Brazil has a tiny advantage in the third place game. They may have had a disappointing defeat on Tuesday, but the numbers still look good.)

But don’t take our word for it...
Now it’s your turn to take a stab at predicting. We have provided an IPython notebook that shows exactly how we built our model and used it to predict matches. We had to aggregate the data that we used, so you can't compute additional statistics from the raw data. However, for the real data geeks, you could try to see how well neural networks can predict the same data or try advanced techniques like principal components analysis. Alternatively, you can try adding your own features like player salaries or team travel distance. We've only scratched the surface, and there are lots of other approaches you can take.

You might also try simulating how the USA would have done if they had beat Belgium. Or how Germany in 2014 would fare against the unstoppable Spanish team of 2010. Or you could figure out whether the USA team is getting better by simulating the 2006 team against the 2010 and 2014 teams.

Here’s how you can do it
We’ve put everything on GitHub. You’ll find the IPython notebook containing all of the code (using pandas and statsmodels) to build the same machine learning models that we've used to predict the games so far. We've packaged it all up in a Docker container so that you can run your own Google Compute Engine instance to crunch the data. For the most up-to-date step-by-step instructions, check out the readme on GitHub.

-Posted by Benjamin Bechtolsheim, Product Marketing Manager

Posted:
Today’s guest blog comes from Jim Totton, Vice President and General Manager, Platform Business Unit at Red Hat

Red Hat Enterprise Linux Atomic Host is now available as a technology preview on Google Compute Engine for customers participating in the Red Hat Enterprise Linux 7 Atomic special interest group (SIG). The inaugural SIG is focused on application containers and encompasses the technologies that are required to create, deploy and manage application containers.

Google and Red Hat actively collaborate on container technologies, approaches and best practices. Both companies are committed to standards for container management, interoperability and orchestration. As a gateway to the open hybrid cloud, application containers enable new possibilities for customers and software providers, including application portability, deployment choice and hyperscale and resilient architectures - whether on-premise or in the cloud.

At Red Hat Summit in April, Red Hat announced our vision for Linux Containers and expanded the Red Hat Enterprise Linux 7 High Touch Beta program to include Red Hat Enterprise Linux Atomic Host – a secure, lightweight and minimal footprint operating system optimized to run Linux Containers. Moving forward, Red Hat will work closely with these participants, with assistance from Google, to support them as they explore application containers. This will help us both gather important requirements and feedback on use cases for these technologies and enable the hybrid cloud for our joint customers.

We also announced today that Red Hat and Google are collaborating to tackle the challenge of how to manage application containers at scale, across hundreds or thousands of hosts. Red Hat will be joining the Kubernetes community and actively contributing code. Earlier today on our blog, we wrote:

Red Hat is embracing the Google Kubernetes project and plans to work to enable it with container management capabilities in our products and offerings. This will enable Red Hat customers to take advantage of cluster management capabilities in Kubernetes, to orchestrate Docker containers across multiple hosts, running on-premise, on Google Cloud Platform or in other public or private clouds. As part of this collaboration, Red Hat will become core committers to the Kubernetes project. This supports Red Hat’s open hybrid cloud strategy that uses open source to enable application portability across on-premise datacenters, private clouds and public cloud environments.

Both Google and Red Hat recognize the importance of delivering containerized applications that are secure, supported and exhibit a chain of trust. Red Hat's Container Certification program, launched in March 2014, supports this commitment and is designed to help deliver containerized applications that “work as intended” to trusted destinations within the hybrid cloud for software partners and end-customers.

Follow the Red Hat Enterprise Linux Blog to stay informed about Red Hat’s work on technologies required to create, deploy, and manage application containers.

-Contributed by Jim Totton, Vice President and General Manager, Platform Business Unit at Red Hat

Posted:
Kubernetes is an open source manager for Docker containers, based on Google’s years of experience using containers at Internet scale. Today, Microsoft, RedHat, IBM, Docker, Mesosphere, CoreOS and SaltStack are joining the Kubernetes community and will actively contribute to the project. Each company brings unique strengths, and together we will ensure that Kubernetes is a strong and open container management framework for any application and in any environment - whether in a private, public or hybrid cloud.

Our shared goal is to allow a broad range of developers to take advantage of container technologies. Kubernetes was built from the ground up as a lean, extensible and portable framework for managing Docker workloads. It lets customers manage their applications the way that Google manages hyper-scale applications like Search and Gmail.

Containers offer tremendous advantages for developers. Predictable deployments and simple scalability are possible because Docker packages all of a workload’s dependencies with the application. This allows for ultimate portability; you can avoid vendor lock-in and run containers in the cloud of your choice. It is just as important that the management framework has the same properties of portability and scalability, and that is what the community will bring to Kubernetes.

We look forward to the contributions of the expanded Kubernetes community:

  • Microsoft is working to ensure that Kubernetes works great in Linux environments in Azure VMs. Scott Guthrie, Executive Vice President of the Cloud and Enterprise group at Microsoft told us, “Microsoft will help contribute code to Kubernetes to enable customers to easily manage containers that can run anywhere. This will make it easier to build multi-cloud solutions including targeting Microsoft Azure.”
  • Red Hat is working to bring Kubernetes to the open hybrid cloud. Paul Cormier, President, Products and Technologies at Red Hat, told us, “Red Hat has a rich history of contributing to and maturing innovative, open source projects. Through this collaboration with Google on Kubernetes, we are contributing to the evolution of cloud computing and helping deliver the promises that container technologies offer to the open hybrid cloud.”
  • IBM is contributing code to Kubernetes and the broader Docker ecosystem to ensure that containers are enterprise-grade, and is working with the community to create an open governance model around the project.
  • Docker is delivering the full container stack that Kubernetes schedules into, and is looking to move critical capabilities upstream and align the Kubernetes framework with Libswarm.
  • CoreOS is working to ensure that Kubernetes can work seamlessly with the suite of CoreOS technologies that support cloud-native application development on any cloud.
  • Mesosphere is actively integrating Kubernetes with Mesos, making the advanced scheduling and management capabilities available to Kubernetes customers.
  • SaltStack is working to make Kubernetes a portable container automation framework that is designed for the reality of the platform-agnostic, multi-cloud world.


You can view the Go source and documentation for Kubernetes on GitHub. We look forward to the contributions of these companies alongside the already vibrant open source community.

- Posted by Urs Hölzle, Senior Vice President