Google Cloud Platform Core Services: GCP Cloud Architect exam preparation

Before we do a deep dive into Google Cloud Platform (GCP) services, let's have a tour to introduce all of the most important core services. This will help us to understand the overall picture in a much better way. It is important to be familiar with all of the core services, not only for the sake of the exam—it will also allow you to choose the best fit for your use case.

Google divides the services into the following logical groups, and they will be the topics we will cover in this article:

• Understanding computing and hosting services
• Exploring storage services
• Getting to know about networking services
• Going through big data services
• Understanding Machine Learning (ML) services
• Learning about identity services

  Exam Tip

  As a cloud architect, you need to understand how most of the services work from a high level. This blog should answer some of the fundamental questions in the exam. Note that Google releases new services very often; however, there is always a delay between a new service release and the exam being updated with new content.

Understanding computing and hosting services

We are given a variety of options when it comes to computing in GCP. Depending on our requirements and flexibility, we can choose from one of the following four options, which we will be looking into in the upcoming sections:

• Infrastructure-as-a-Service (IaaS): Google Compute Engine (GCE)
• Container-as-a-Service (CaaS): Google Kubernetes Engine (GKE)
• Platform-as-a-Service (PaaS): Google App Engine (GAE)
• Function-as-a-Service (FaaS): Cloud Functions

  Exam Tip

  Note that there are additional compute options that might not yet appear on the exam but were announced in 2019. GKE On-Prem is a GKE service that can be installed on your local environment and managed from your Google Console. Cloud Run is an FaaS offering that allows you to define containers that will listen for HTTP requests. This allows you to use languages that are not supported by Cloud Functions. To read more about these services, check the Further reading links.

• Fully managed container platform: Cloud Run

The choice we make may depend on several factors. For example, do we need full control over our infrastructure, or do we want a fully managed service? Starting with Compute Engine, we have control over the Virtual Machine (VM) instance. This gives us the most flexibility but also implies that we need to take care of the stack above it. With Cloud Functions, we are very much constrained by the supported programming languages (JavaScript, Python, Node.js, Java, .NET, Ruby, PHP, and Go). The advantage of Cloud Functions is that you don't need to worry about infrastructure and scaling. You only concentrate on developing your functions. If you need to use a language that is not supported by Cloud Functions, you will not be able to use it.

The computing options in GCP are as shown in the following diagram:

Figure 3.1 – Computing options

Let's have a look at each of the compute options and see what is managed by Google for us against the flexibility that we are given:

• GCE: GCE is an IaaS offering. It allows the most flexibility as it provides compute infrastructure to provision VM instances. This means that you have control of the virtualized hardware and operating system. Note, this can be limited to available machine types. You can use standard GCP images or your own custom image. You can control where your VMs and storage are located in terms of regions and zones. You have granular control over the network, including firewalls and load balancing. With the use of an instance group, you can autoscale your control and your capacity as needed. Compute Engine is suitable in most cases but might not be an optimal solution.
• GKE: GKE is a CaaS offering. It allows you to create Kubernetes clusters on demand, which takes away all of the heavy lifting of installing the clusters yourself. It leverages Compute Engine for hosting the cluster nodes, but the customer does not need to bother with the infrastructure and can concentrate on writing the code. The provisioned cluster can be automatically updated and scaled. The GCP software-defined networks are integrated with GKE and allow users to create network objects, such as load balancers, on demand when the application is deployed. Several services integrate with GKE, such as Artifact Registry, which allows you to store and scan your container images.
• App Engine: App Engine is a PaaS offering. It allows you to concentrate on writing your code, while Google takes care of hosting, scaling, monitoring, and updates. It is targeted at developers who do not need to understand the complexity of the infrastructure. GAE offers two types of environments, as follows:
  • Standard: With sets of common languages supported, including Python, Go, Java, Node.js, PHP, Ruby, and Go.
  • Flexible: Even more languages, with the possibility of creating a custom runtime. With a flexible environment, you lose some out-of-the-box integration, but you gain more flexibility.

GAE is tightly integrated with GCP services, including databases and storage. It allows versioning of your application for easy rollouts and rollbacks:

• Cloud Functions: Cloud Functions is a FaaS offering. It allows you to concentrate on writing your functions in one of the supported languages. It is ideal for executing simple tasks for data processing, mobile backends, and IoT. This service is completely serverless and all of the layers below it are managed by Google. The functions can be executed using an event trigger or HTTP endpoint.
• Cloud Run: Brings together the simplicity of FaaS and portability of CaaS. It allows you to develop and deploy self-scaling containerized applications on a fully managed serverless platform. It is compatible with Knative so you can move your workloads to any environment that can run Kubernetes in the cloud or on-premises.
• Anthos: Anthos is a modern application management platform that provides a consistent development and operations experience for cloud and on-premises environments. Anthos is not a compute option itself but allows you to run Google Kubernetes Engine and Cloud Run on Anthos in multi-cloud and hybrid environments.
• Google Cloud VMware Engine (GCVE) is a fully managed native VMware Cloud Foundation software stack hosted in GCP. It allows you to accelerate the move to GCP by lifting and shifting your VMs hosted on vSphere into Google Cloud as is.

Now that we have studied compute services, let's have a look at storage services in the next section.

Exploring storage services

Storage is an essential part of cloud computing as it saves the data and state of your applications. GCP offers a wide variety of storage, from object storage to managed databases. The different storage services that we will be looking at are as follows:

• Cloud Storage: Cloud Storage is a fully managed, object-oriented storage service with a virtually infinite capacity. It allows the creation of buckets that store your data and allow access through APIs and tools such as gsutil. It comes with different storage classes and locations to best suit your needs in terms of how often your data will be accessed and where it should be located. Keep in mind that the price differs for each tier. Making a conscious decision will allow you to cut costs. You can choose from the following options:
  • Standard: The highest availability in multiple geolocations
  • Nearline: For data accessed less than once a month
  • Coldline: Very low cost for data accessed less than once a quarter
  • Archive: The lowest cost for data accessed less than once a year

    Important Note

    Previously, Google Cloud Storage offered slightly different storage classes than the previously mentioned ones. The exam may not have updated this as yet, so it is important to also know the older options, as follows:

    Multi-regional: The highest availability in multiple geolocations

    Regional: High availability with fixed locations

    Nearline: Low-cost, for data accessed less than once a month

    Coldline: The lowest cost for backup and disaster recovery

With Cloud Storage, you do not need to worry about running out of capacity. The following list informs us of other Google Cloud Storage options:

• Cloud Filestore: Filestore is a managed file storage service. It allows users to provision a Network Attached Storage (NAS) service that can be integrated with GCE and GKE. It comes with two performance tiers—standard and premium, which offer different Input/Output operations Per Second (IOPS) and throughputs.
• Cloud SQL: Cloud SQL is a fully-managed relational database service for either a MySQL, PostgreSQL, or SQL Server database. It offers data replication, backups, data exports, and monitoring. It is ideal when you need to move your current instances from on-premises and want to delegate the maintenance of the database to Google.
• Cloud Datastore: Cloud Datastore is a fully managed non-SQL database. It is ideal for applications that rely on highly available structured data at scale. The scaling and high availability are achieved with a distributed architecture and are abstracted from the user. There is only one database available per project. Cloud Datastore offers SQL-like language to query your data. It has been superseded by Cloud Firestore.
• Cloud Firestore: Cloud Firestore is the next generation of Cloud Datastore with several enhanced features. It can run in Native or Datastore mode. The former is compatible with Cloud Datastore. Google has already started moving all Datastore clients to Cloud Firestore without any downtime or any user intervention. All new projects should be created in Cloud Firestore instead of Datastore.
• Cloud Spanner: Cloud Spanner is a fully managed, globally distributed database service. It offers the strong consistency of a relational database with non-relational database scaling capabilities. Users can define a schema and leverage industry-standard American National Standards Institute (ANSI) 2011 SQL. It delivers high-performance transactions, with a 99.999% availability Service-Level Agreement (SLA), meaning there is almost no downtime. Cloud Spanner is aimed at use cases such as financial trading, insurance, global call centers, telecoms, gaming, and e-commerce. Global consistency makes it ideal for globally accessible applications.
• Bigtable: Bigtable is a fully managed, massive scale, NoSQL database service with a consistent sub-10ms latency for large analytical and operational workloads. It is used by Google to deliver services such as Gmail and Google Maps. It is ideal for fintech, IoT, and ML storage use cases. It integrates easily with big data product families such as Dataproc and Dataflow. It is based on open source Apache HBase, enabling the use of its API. The cost of Bigtable is much higher than Datastore, so the database should be chosen with great care.
• Custom databases: You can also choose to use Compute Engine to install a database of your choice, such as MongoDB; however, that would be an unmanaged service.
• Persistent Disks: These are durable network storage devices that can be accessed by our instances as if they were physical disks. They are located independently from our VM instances and can be detached and moved to keep data safe even after a VM is deleted.

With this, we conclude our look at storage services; let's have a look at networking services in the next section.

Getting to know about networking services

GCP networking is based on Software-Defined Networks (SDNs), which allow users to deliver all networking services programmatically. All of the services are fully managed, leaving users with the task of configuring them according to their requirements. The networking services that we will be looking at are as follows:

• Virtual Private Cloud (VPC): The VPC is the foundation of GCP networking. Projects can contain multiple VPC networks. Unless you create an organizational policy that prohibits it, new projects start with a default network (an auto mode VPC network) that has one subnetwork (subnet) in each region. You can think of it as a cloud version of a physical network. Each VPC network can contain one or more regional subnets. A VPC creates a global logical boundary that allows communication between resources within the same VPC network, subject to applicable network firewall rules. To allow communication between VPCs, traffic needs to traverse the internet or use VPC peering.
• Load balancer: A load balancer allows the distribution of traffic between your workloads. They are available for GCE, GAE, and GKE. For GCE, you can choose from load balancers with global or regional scopes. The choice will also depend on the network type. The following load balancers are available:
  • HTTP(S) load balancer
  • SSL proxy
  • TCP proxy
  • External network load balancer
  • Internal TCP/UDP load balancer
  • Internal HTTP(S) load balancer
• Cloud Router: Cloud Router is a service that allows you to dynamically exchange routes between VPC and on-premises networks by using Border Gateway Protocol (BGP): https://www.wikipedia.org/wiki/Border_Gateway_Protocol. It eliminates the need for the creation of static routes.
• Virtual Private Network (VPN): VPNs allow a connection between your on-premises network and GCP VPC through an IPsec tunnel over the internet. Only site-to-site VPNs are supported. Traffic in transit is encrypted. Both static and dynamic routing are supported, with the latter requiring a cloud router. Using a VPN should be considered an initial method of connecting your environment to GCP as it entails the lowest cost. If there are low-latency, high-reliability, and high-bandwidth requirements, then Cloud Interconnect should be considered.
• Cloud Interconnect: If there is a need for low latency and a highly available connection between your on-premises and Google Cloud VPC networks, then interconnect should be considered. In this case, the traffic does not traverse the internet. There are two interconnect options, which are as follows:
  • Dedicated Interconnect: 10 or 100 Gbps piped directly to a Google datacenter
  • Partner Interconnect: 50 Mbps-50 Gbps piped through a Google partner
• Cloud DNS: Cloud DNS is a managed DNS service with a 100% SLA. It resolves domain names into corresponding IP addresses. It can be used to publish and manage millions of DNS zones and records with ease. Cloud DNS can also host private zones accessible only from one or more VPC networks that you specify.
• Cloud Content Delivery Network (CDN): Cloud CDN is a service that allows the caching of HTTP(S) load balanced content, from various types of backends, including Cloud Storage buckets. Caching reduces content delivery time and cost. It can also protect you from a Distributed Denial-of-Service (DDoS) attack. Data is cached on Google's globally distributed edge points. On the first request, when content is not cached, data is retrieved from a backend origin. The subsequent requests for the same data will be served directly from the cache until the expiration time is reached.
• Cloud NAT: Cloud NAT is a regional service that allows VMs without external IPs to communicate with the internet. It's a distributed, software-defined managed service that can be configured to automatically scale the number of NAT IP addresses that it uses. It works with both GCE and GKE. It is a better alternative for NAT instances that need to be managed by users.
• Firewall: GCP Firewall is a service that allows for micro-segmentation. Firewall rules are created per VPC and can be based on IPs, IP ranges, tags, and service accounts. Several firewall rules are created by default but can be modified.
• Identity-Aware Proxy (IAP): IAP is a service that replaces the VPN when a user is working from an untrusted network. It controls access to your application based on user identity, device status, and IP address. It is part of Google's BeyondCorp (https://cloud.google.com/beyondcorp) zero trust security model.
• Cloud Armor: Cloud Armor is a service that allows protection against infrastructure DDoS attacks using Google's global infrastructure and security systems. It integrates with global HTTP(S) load balancers and blocks traffic based on IP addresses or ranges. The preview mode allows users to analyze the attack pattern without cutting off regular users.

Phew! We have covered a lot about networking services. Now, let's look at big data services in the following section.

Going through big data services

Big data services enable the user to process large amounts of data to provide answers to complex problems. GCP offers many services that tightly integrate to create an End-to-End (E2E) data analysis pipeline. These services are as follows:

• BigQuery: BigQuery is a highly scalable and fully managed cloud data warehouse. It allows users to perform analytics operations with built-in ML. BigQuery is completely serverless and can host petabytes of data. The underlying infrastructure scales seamlessly and allows parallel data processing. The data can be stored in BigQuery Storage, Cloud Storage, Bigtable, Sheets, or Google Drive. The user defines datasets containing tables. BigQuery uses familiar ANSI-compliant SQL for queries and provides ODBC and JDBC drivers. Users can choose from two types of payment models—one is flexible and involves paying for storage and queries, and the other involves a flat rate with stable monthly costs. It is ideal for use cases such as predictive analysis, IoT, and log analysis, and integrates with GCP's big data product family.
• Pub/Sub: This is a fully managed asynchronous messaging service that allows you to loosely couple your application components. It is serverless with global availability. Your application can publish messages to a topic or subscribe to it to pull messages. Pub/Sub also offers push-based delivery of messages as HTTP POST requests to Webhooks.
• Dataproc: Dataproc is a fully managed Apache Spark and Hadoop cluster service. It allows users to create clusters on demand and use them only when data processing is needed. It is billed per second. It allows users to move already existing, on-premises clusters to the cloud without refactoring the code. The use of pre-emptible instances can further lower the cost.
• Cloud Dataflow: Cloud Dataflow is a fully managed service for processing data in streams and batches. It is based on open source Apache Beam, is completely serverless, and offers almost limitless capacity. It will manage resources and job balancing for the user. It can be used for use cases such as online fraud analytics, IoT, healthcare, and logistics.
• Dataprep: This is a tool that can be used to perform data preparation and visualization. We can explore and transform data without any coding skills being required. Data can be interactively prepared for further analysis.
• Datalab: Datalab is a tool built into Jupyter (formerly IPython) that allows users to explore, analyze, and transform data. It also allows users to build ML data models and leverages Compute Engine.
• Data Studio: This is a tool that allows you to consume data from sources and visualize it in the form of reports and dashboards.
• Cloud Composer: This is a fully managed service based on open source Apache Airflow. It allows you to create and orchestrate big data pipelines.
• Data Fusion: This is a fully managed enterprise data integration service. It provides a UI that allows you to build and manage pipelines to clean, prepare, blend, transfer, and transform data.

Finally, let's study ML services in the next section.

Understanding machine learning services

One of the strongest points of Google is its long-term experience with Machine Learning (ML). GCP offers several services around ML. You can choose between a pre-trained model or train a model yourself. The various services included under ML are as follows:

• Pretrained APIs: ML APIs are services that allow you to leverage several pre-trained models, enabling you to analyze a video. Currently, the following APIs are available:
  • Google Cloud Video Intelligence
  • Google Cloud Speech
  • Google Cloud Vision
  • Google Cloud Natural Language
  • Google Cloud Translation
  • Google Recommendations AI

The following list of models can be used without any background knowledge of how they work. As an example, we can analyze text for sentiment analysis:

• AutoML: AutoML is a service that can be used by developers to train models without having extensive knowledge of data science. As an example, by providing labeled samples to AutoML, it can be trained to recognize objects that are not recognizable by the Vision API. The following are the labeled samples of AutoML:
  • AutoML Translation
  • AutoML Natural Language
  • Video Intelligence
  • AutoML Tables
• Dialogflow: This is a service that allows you to build conversation applications that can interact with human beings. The interface can interact with many compatible platforms, such as Slack or Google Assistant. It can also integrate with Firebase functions to integrate with third-party platforms using common APIs.
• AI Platform: This was a fully managed service to allow the E2E development of a machine learning model. Before it went to general availability, Google released Vertex AI and deprecated AI Platform.
• Vertex AI: This is a unified machine learning platform to build, deploy, and scale AI models. It integrates AutoML and AI Platform together into a unified API, client library, and user interface. With Vertex AI, you can perform both AutoML training and custom training. For both of those options, you can save models, deploy models, and request predictions using Vertex AI.

ML services will be discussed further in This Post, Putting Machine Learning to Work. In the next section, we will introduce Google's identity and access management services.

Learning about identity services

Identity and Access Management (IAM) is one of the most important aspects of any cloud. It allows you to control who has access to the cloud but can also provide identity services to your applications. In short, this is achieved by a combination of roles and permissions. The roles are assigned to either users or groups. Let's have a look at the options we have in GCP:

• IAM: IAM allows the GCP admin to control authorization to GCP services. Administrators can create roles with granular permissions. Roles can then be assigned to users, or preferably, a group of users.
• Cloud Identity: Cloud Identity is an Identity-as-a-Service (IDaaS) offering. It sits outside of GCP but can be easily integrated with GCP. It allows you to create organizations, groups, and users, and manage them centrally. If you already have an existing user catalog, you can synchronize it with Cloud Identity.

IAM is very important to understand because it underpins how your GCP architecture will be created. We will discuss this more in This Post, Security and Compliance.

Summary

In this post, we learned about GCP services and gathered them in specific groups: compute and hosting, storage, networking, big data, ML, and identity services. This allowed us to get a broad overview of what GCP offers us. That should give you a little bit more confidence in what GCP actually is, but you probably also understand that there is quite some work ahead of us. But don't worry, we'll get there together!

Once you've finished going through this article, let's switch to This Post, Working with Google Compute Engine, where we will finally get some hands-on experience with deploying our first services to GCP. This is getting exciting!

Further reading

You can refer to the following links to gain more information:

• GCP core services and products: https://cloud.google.com/products/.
• Cloud Run: https://cloud.google.com/run/
• GKE On-Prem: https://cloud.google.com/gke-on-prem/