In a nutshell: How OpenStack works

In a nutshell: How OpenStack works?

In a nutshell: How OpenStack works

Explaining what OpenStack is and how it works in a short post is a unrealistic idea, nevertheless I’ll try to write a brief description of present components in an standard environment and the way they communicate to give life to what is known as cloud

What is OpenStack?

OpenStack is an IaaS cloud computing project that is free open-source software.

Its mission is to provide a flexible solution for both public and private clouds of any size, and for this matter two basic requirements are considered: clouds must be simple to implement and massively scalable.

To meet these principles OpenStack is divided into different components that work together. This integration is achieved through application programming interfaces – APIs – offered and consumed by each service.

With these APIs, services can communicate with each other and also allows a service to be replaced by another with similar characteristics, only if the form of communication is respected. That is, OpenStack is extensible and meets the needs of those who wish to implement it.

Structure

Relationships between OpenStack services can be seen conceptually as follows

OpenStack Conceptual View

OpenStack Folsom Conceptual View – by Ken Pepple

This is a simplified view of the architecture, assuming that all the services are used in the most standard configuration. Nor illustrates how the cloud consumers can interact with it.

Following components can be seen on the graph

  • “Horizon” Dashboard provides an end user and administrator interface to other services. It is the service I’m currently working ;)
  • “Nova” Compute retrieves images and associated metadata, and transforms user requests on virtual machines.
  • “Neutron” Network provides virtual networks as a service between devices managed by other OpenStack services, such as a virtual machine from Nova. Allows users to create their own networks and then link them to the devices of their choice.
  • “Cinder” Block Storage provides persistent storage for VMs hosted on the cloud.
  • “Glance” Image provides a catalog and a repository for images.
  • “Swift” Object Store provides object storage. This is not a file system, is more like a container that can store files and retrieve them later.
  • “Keystone” Identity provides authentication and authorization for all OpenStack services, and a catalog of these services of a particular cloud.
OpenStack Software Diagram

OpenStack Software Diagram – by OpenStack

General info: main components of each service

Horizon – Dashboard
  • Is a Django web application, a web framework for perfectionists ;) Those familiar with this framework will find Horizon code easy to understand.
  • It is implemented through mod_wsgi in Apache.
  • mod_wsgi implements an Apache easy to use module, able to accommodate any application developed in Python that supports WSGI interface.
  • Web Server Gateway Interface WSGI is a specification for application servers and web servers to communicate with web applications.
    • WSGI by Pylons

    WSGI middleware – by Pylons

  • The code is divided into reusable modules with logic, interaction with APIs, and presentation, to make customization in different sites easier.
  • It also has a small database, SQLite3 by default, for some options, but most of the data are provided by the other services.
  • Horizon is an implementation of the Dashboard, not the Dashboard itself. Different implementations can be made to fit the user’s needs.
Nova – Compute
OpenStack Compute by Rackspace

Openstack Compute – by Rackspace

  • nova-api client accepts and responds to end user calls.
  • Virtualization is managed by nova-compute. It creates/terminates VMs instances via selected hypervisor APIs.
  • Storage is controlled by nova-volume (now replaced by Cinder). Manages creation, attaching and detaching of persistent volumes to compute instances.
  • Networking is administered by nova-network (now replaced by Neutron). Accept networking tasks and then manipulate the network.
  • Scheduling is performed by nova-schedule. Take VM requests from the queue and determine where it should run.
  • The queue, RabbitMQ by default, is the central node for message passing between daemons.
  • It also has a database that store most of the build-time and run-time state.
  • nova-consoleauth, nova-novncproxy, nova-console allows end users to access their virtual instances through a proxy.
  • When starting an instance a set of virtual resources known as a flavor must be selected. Additional resources such as persistent volume storage and public IP address may be added.

Compute

Swift – Object Storage
Object Storage by Rackspace

OpenStack Object Storage – by Rackspace

Before talking about Swift components…

What an object is?

This notion is fundamental to understand this service since the administration of objects is it main objective.

An object is an entity containing data, but unlike files we usually work, objects are not organized in a hierarchy

Every object exists at the same level in a flat address space called a storage pool. One object cannot be placed inside another object.

Both files and objects have metadata associated with the data they contain, but objects are characterized by their extended metadata. Each object is assigned a unique identifier which allows a server or end user to retrieve the object without needing to know the physical location of the data. This approach is useful for automating and streamlining data storage in cloud computing environments.

Object storage is often compared to valet parking at an upscale restaurant. When a customer uses valet parking, he exchanges his car keys for a receipt. The customer does not know where his car will be parked or how many times an attendant might move the car while the customer is dining. In this analogy, a storage object’s unique identifier represents the customer’s receipt.

Talking about Swift again, we can mention

  • Proxy server accepts incoming requests, like files to upload, modifications to metadata or container creation; it also serve files and container listing.
  • Accounts server manage accounts defined with the object storage service.
  • Container servers manage a mapping of containers, folders, within the object store service.
  • Object servers manage actual objects, files, on the storage nodes.
  • Also replication services run to provide consistency and availability across the cluster, audit and update.

Object Storage Swift

Glance – Image Store

In this case also…

What an image is?

An image is a single file containing the complete contents and structure of a storage medium. It is an identical copy of a file system.

Images are frequently used as a distribution medium for operating systems and instances (one particular execution time) of them. The latter are typically known as snapshots.

In other words, snapshots are running instances that can be obtained by creating a new image based on the current state of the disk of a particular instance.

Inside Glance we will find,

  • glance-api accept calls for image discovery, retrieval and storage.
  • The storage registry glance-registry processes and retrieves metadata about images.
  • It has a database to store image metadata.
  • Also replication services run to provide consistency and availability across the cluster, audit and update.
Keystone – Identity
  • keystone handles API requests as well as providing single point of integration for policy, configurable catalog, token and authentication (identity services)
  • Every Keystone function has a pluggable backend which allows different ways to use the particular service. It supports standard backends like LDAP or SQL, and KVS (Key-Value Stores).
Neutron – Network
  • neutron-server accept API requests and route them to the correct neutron plugin.
  • Plugins and agents perform actual actions, like plug/unplug ports, creating networks and subnets and IP addresing.
  • It also has a message queue to route info between neutron-server and various agents.
  • It has a database to store networking state for particular plugins.

Network Neutron

Cinder – Block Storage
  • Cinder API allows for manipulation of volumes, volumes types and snapshots. cinder-api accepts requests and routes them to cinder-volume for action.
  • cinder-volume reacts reading or writing to the cinder database to maintain state, interacts with other processes (like cinder-scheduler) through a message queue and directly on block storage providing hardware or software.
  • cinder-scheduler picks the optimal block storage node to create the volume on.
  • The messages queue route information between Cinder processes.
  • A database store volumes state.

All these components and how they relate are shown in their most simple configuration in the following graph

OpenStack Logical View

OpenStack Folsom Logical View – by Ken Pepple

Probably you will want to open the image in another tab … There are too many relatioships! And as I said, it is a simplified view.

I’m sure that many things remain pending – I am aware that the description of Keystone is pretty poor, and I will improve it! -, but I hope at least to have prepared the ground for the next posts, more specific and with more technical issues.

Via | OpenStack Manuals, Rackspace, TechTarget and Techrepublic

Pictures without caption are from OpenStack.

Comments (46)

  2. Peter - Reply

    great overview article, but one remark about figure 1: The labels on the arrows are “oriented” from one end to the other, which is usually backed by the use of oneway-arrows as transitions. Here one has to interpret the direction of the arrow out of the content context. From a diagram pov I think that’ an error.

    Nevertheless: great overview alltogether.

    • vkmc - Reply

      I understood the two-way arrows like both services can communicate with each other in a full-duplex style. But you may be right, it can be confusing… using two independent arrows seems clearer.

      I’ll try to fix it!

      Thanks for the feedback Peter!

    • Ken Pepple - Reply

      Hi Peter — I created figure 1 (openstack conceptual architecture). The arrows are meant to show conceptual interactions not direct resource / consumer relationships. Having said that, I’ll try to make the Grizzly set of pictures more clear and maybe using the arrows as you suggest might be a good idea.

      • vkmc - Reply

        Ken, thanks for explaining how the diagrams should be understood. I guess I get the most of them, but having your guidance is even better.

        About that, excellent work :) Your post gave me the tools needed to break the ice with OpenStack and get involved with the code in an easier way.

        I’m looking forward to Grizzly edition!

  3. Pingback: OpenStack Community Weekly Newsletter (Feb 1 – 8) » The OpenStack Blog

  4. Sriram Subramanian - Reply

    Nice one, clear and concise, very easy to understand!

    Could you please make the diagrams for Nova-Compute, Horizon and Swift clickable/ expandable. It will be nice to see them little bigger. I have some comments on grammar – may be via email is better?

    • vkmc - Reply

      Thanks a lot Sriram!

      I’d make those diagrams clickable :)

      And, please, add your suggestions here or by email, either way I’ll appreciate that a lot.

  6. Stavros - Reply

    Great job writing this overview. It’s a great document that explains things both in detail and from a high-level overview. I like the way you break things down. I will use this when explaining to different people how things relate and break down as it illustrates that easily.

    Great work and no worries about the language. For someone speaking English as a second language, you did awesome!

    • vkmc - Reply

      Thanks a lot for your words Stavros! I’m glad you found this post useful – that’s my main goal when writing them :) -, and I hope this give you an extra tool to guide people’s learning.

      Good to know I’m getting better with my English :) I’ll still learning though!

      Thanks again

  7. Pingback: OpenStack in a nutshell | GCU-Squad!

  8. Pingback: How OpenStack works? | Brent Sordyl's blog

  10. Pingback: Links & reads for 2013 Week 14 | Martin's Weekly Curations

  11. Pingback: Technology Short Take #31 - blog.scottlowe.org - The weblog of an IT pro specializing in virtualization, storage, and servers

  12. Pingback: Putting the H in OpenStack | DisruptingIT

  13. Ashish Barot - Reply

    Really very good post, which helps me to understand in more detail.
    Please keep sharing with us, we will appreciate your efforts.

    Best Regards,
    Ashish Barot.

    • vkmc - Reply

      Thanks Ashish! I’m glad it was useful for you :) I have been a little busy this lasts weeks, but I’ll update the blog soon! Cheers

  14. Pingback: Technology Short Take #31 | Strategic HR

  15. Pingback: How OpenStack Works | Xizeng Mao's Blog

  16. Hugo Katchen - Reply

    Pretty nice post. I simply stumbled upon your weblog and wanted to
    say that I’ve really enjoyed browsing your weblog posts.
    In any case I will be subscribing in your rss feed and I’m hoping you write once more soon!

    • vkmc - Reply

      Hi Hugo! I’m glad to hear you enjoyed my posts. Now I’m having some more time, so I’ll get back soon with new stuff. Thanks for passing by!

  18. Pingback: Finding an OpenStack Mentor | Just Write Click

  23. somshekar - Reply

    Really good overview, got my concepts cleared, will there be any post for one complete cycle interaction between any api with web framework, web server ex a simple nova list in backend that will be helpful,
    We would appreciate it, it will help developer on this side, please excuse me IF have asked for more.

    • vkmc - Reply

      Hi! It sounds very good… I’m planning to do something alike for the OpenStack Trove project. Thanks for passing by :)

  27. Pingback: OpenStack – Physical World

  28. Pingback: OpenStack Komponenten - ScaleUp Technologies

Leave a Comment

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>