https://www.youtube.com/watch?v=zk42zzc_38o
Introduction to OpenStack Trove: A Multi-database Deployment with MongoDB and MySQL
Tuesday, August 26, 2014
Saturday, August 23, 2014
OpenStack : Neutron : Neutron problem and solution
Problem :
in /var/log/neutron/openvswitch-agent.log
2014-08-23 14:46:58.105 4456 TRACE neutron.plugins.openvswitch.agent.ovs_neutron_agent Timeout: Timeout while waiting on RPC response - topic: "q-plugin", RPC method: "security_group_rules_for_devices" info: "<unknown>"
[openstack@novo ~(keystone_admin)]$ neutron port-list
Connection to neutron failed: Maximum attempts reached
Solution :
[root@novo sbin]# service neutron-server restart
Stopping neutron: [FAILED]
Starting neutron: [ OK ]
After that :
in /var/log/neutron/openvswitch-agent.log
2014-08-23 14:47:00.384 4456 INFO neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-45eca3db-5fa3-45da-b439-526819e7a010 None] Configuration for device 24110dfe-84f9-4dca-beb0-e17cf50b291f completed.
[openstack@novo ~(keystone_admin)]$ neutron port-list
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| id | name | mac_address | fixed_ips |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
| 24110dfe-84f9-4dca-beb0-e17cf50b291f | | fa:16:3e:b4:df:f0 | {"subnet_id": "4de1afb9-ed1f-4c32-9bbb-20eec6073963", "ip_address": "10.0.0.2"} |
| 36b6c681-4032-495e-a799-ab05a4d35249 | | fa:16:3e:29:ed:8c | {"subnet_id": "4de1afb9-ed1f-4c32-9bbb-20eec6073963", "ip_address": "10.0.0.3"} |
| 426f579f-5506-4229-99ff-6840e4b38dea | | fa:16:3e:a4:b2:07 | {"subnet_id": "4de1afb9-ed1f-4c32-9bbb-20eec6073963", "ip_address": "10.0.0.8"} |
| 5e313f5f-0603-41d2-b652-ff1f1c84dece | | fa:16:3e:85:28:cd | {"subnet_id": "96a64200-a1fb-40e8-b888-c16bbe70609a", "ip_address": "172.24.4.235"} |
| b972f13b-5f89-4e93-a46a-b797aca9646e | | fa:16:3e:b0:e4:eb | {"subnet_id": "96a64200-a1fb-40e8-b888-c16bbe70609a", "ip_address": "172.24.4.236"} |
| c322386e-cea0-46a4-a520-8c3123f9ea2f | | fa:16:3e:01:ff:59 | {"subnet_id": "96a64200-a1fb-40e8-b888-c16bbe70609a", "ip_address": "172.24.4.231"} |
| cad3ee33-844e-4a1a-be7c-95751eb71600 | | fa:16:3e:56:9f:44 | {"subnet_id": "4de1afb9-ed1f-4c32-9bbb-20eec6073963", "ip_address": "10.0.0.1"} |
| d7b68106-4785-419e-89c8-c553ddfc92f3 | | fa:16:3e:03:3f:5c | {"subnet_id": "4de1afb9-ed1f-4c32-9bbb-20eec6073963", "ip_address": "10.0.0.9"} |
| f7ae4cd8-393d-47e1-a875-20b30adb2f4f | | fa:16:3e:48:d9:c0 | {"subnet_id": "96a64200-a1fb-40e8-b888-c16bbe70609a", "ip_address": "172.24.4.232"} |
+--------------------------------------+------+-------------------+-------------------------------------------------------------------------------------+
Wednesday, August 20, 2014
ISTC for Big Data : Michael Stonebraker
http://istc-bigdata.org/index.php/istc-for-big-data-michael-stonebraker/
Tuesday, August 19, 2014
Google Bigtable paper
http://research.google.com/archive/bigtable.html
Bigtable: A Distributed Storage System for Structured Data
Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber
Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, and Robert E. Gruber
Abstract
Bigtable is a distributed storage system for managing structured data that is designed to scale to a very large size: petabytes of data across thousands of commodity servers. Many projects at Google store data in Bigtable, including web indexing, Google Earth, and Google Finance. These applications place very different demands on Bigtable, both in terms of data size (from URLs to web pages to satellite imagery) and latency requirements (from backend bulk processing to real-time data serving). Despite these varied demands, Bigtable has successfully provided a flexible, high-performance solution for all of these Google products. In this paper we describe the simple data model provided by Bigtable, which gives clients dynamic control over data layout and format, and we describe the design and implementation of Bigtable.
To appear in:
OSDI'06: Seventh Symposium on Operating System Design and Implementation,
Seattle, WA, November, 2006.
OSDI'06: Seventh Symposium on Operating System Design and Implementation,
Seattle, WA, November, 2006.
Download: PDF Version
Xen vs. KVM
Source :
http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance-zones-kvm-xen/
With Xen, the hypervisor performs CPU scheduling for the domains, and then each domain has its own OS kernel for thread scheduling. The hypervisor supports different CPU scheduling classes, including Borrowed Virtual Time (BVT), Simple Earliest Deadline First (SEDF), and Credit-Based. The domains use the OS kernel scheduler, and whatever regular scheduler classes and policies they provide.
The extra overhead of multiple schedulers costs performance. Having multiple schedulers can also create complex issues with how they interact, adding CPU latency in the wrong situations. Debugging this can be very difficult, especially since the Xen hypervisor is running out of reach of the usual OS performance tools (try xentrace instead).
Sending I/O via the I/O proxy processes (which are usually qemu) involves context-switching and more overhead. There has been lots of work to minimize this, including shared memory transports, buffering, I/O coalescing, and paravirtualization drivers.
With KVM, the hypervisor is a kernel module (kvm) which is scheduled by the OS scheduler. It can be tuned using the usual OS kernel scheduler classes, policies and priorities. The I/O path takes fewer steps than Xen. (The original Qumranet KVM paper described it as five steps vs ten, although this description isn’t including paravirtualization.)
Source :
https://major.io/2014/06/22/performance-benchmarks-kvm-vs-xen/
http://dtrace.org/blogs/brendan/2013/01/11/virtualization-performance-zones-kvm-xen/
With Xen, the hypervisor performs CPU scheduling for the domains, and then each domain has its own OS kernel for thread scheduling. The hypervisor supports different CPU scheduling classes, including Borrowed Virtual Time (BVT), Simple Earliest Deadline First (SEDF), and Credit-Based. The domains use the OS kernel scheduler, and whatever regular scheduler classes and policies they provide.
The extra overhead of multiple schedulers costs performance. Having multiple schedulers can also create complex issues with how they interact, adding CPU latency in the wrong situations. Debugging this can be very difficult, especially since the Xen hypervisor is running out of reach of the usual OS performance tools (try xentrace instead).
Sending I/O via the I/O proxy processes (which are usually qemu) involves context-switching and more overhead. There has been lots of work to minimize this, including shared memory transports, buffering, I/O coalescing, and paravirtualization drivers.
With KVM, the hypervisor is a kernel module (kvm) which is scheduled by the OS scheduler. It can be tuned using the usual OS kernel scheduler classes, policies and priorities. The I/O path takes fewer steps than Xen. (The original Qumranet KVM paper described it as five steps vs ten, although this description isn’t including paravirtualization.)
Source :
https://major.io/2014/06/22/performance-benchmarks-kvm-vs-xen/
Oracle VM Architecture
Source is here
Client Applications: Various user interfaces to Oracle VM Manager are provided, either via the graphical user interface (GUI) accessible using a web-browser; the command line interface (CLI) accessible using an SSH client; custom built applications or scripts that use the Web Services API (WS-API); or external applications, such as Oracle Enterprise Manager, or legacy utility scripts that may still make use of the legacy API over TCPS on port 54322. The legacy API is due to be deprecated in the near future and applications that are using it must be updated to use the new Web Services API instead. All communications with Oracle VM Manager are secured using either a key or certificate based technology.
Oracle VM Manager: Used to manage Oracle VM Servers, virtual machines, and resources. It is comprised of a number of subcomponents, include a web browser-based user interface; and a command line interface (CLI) allowing you to manage your infrastructure directly from the command line either via external scripts or by running manual command sequences. Each of these interfaces runs as a separate application to the Oracle VM Manager core and interfaces with this using the Web Services API.
The Oracle VM Manager core is an Oracle WebLogic Server application running on Oracle Linux. The user interface uses the Application Development Framework (ADF) application, providing a common look and feel, in line with other Oracle web-based applications. Oracle VM Manager can be on a standalone computer, or part of a virtual machine running on an instance of Oracle VM Server. While Oracle VM Manager core and the Oracle VM Manager GUI are both WebLogic applications, they are separate applications, even though they share the same process space.
Oracle VM Manager communicates with each Oracle VM Server via the Oracle VM Agent, using XML-RPC over HTTPS on port 8899. Actions on servers that are initiated within Oracle VM Manager are triggered using this method. The Oracle VM Agent on each Oracle VM Server is equally able to send notifications, statistics and event information back to Oracle VM Manager. Actions within Oracle VM Manager triggered by Oracle VM Agent are achieved using the Web Services API exposed by Oracle VM Manager and are secured using HTTPS.
While Oracle VM Manager is a critical component for configuration actions within the Oracle VM infrastructure, the virtualized environment can continue to function properly even if Oracle VM Manager experiences downtime. This includes the ability to maintain high availability and to perform live migration of virtual machines
Oracle VM Manager Database: Used by Oracle VM Manager core to store and track configuration, status changes and events. Oracle VM Manager uses a MySQL Enterprise database that is bundled in the installer and which runs on the same host where Oracle VM Manager is installed. The database is configured for the exclusive use of Oracle VM Manager and must not be used by any other applications. The database is automatically backed up on a regular schedule, and facilities are provided to perform manual backups as well.
Oracle VM Server: A managed virtualization environment providing a lightweight, secure, server platform which runs virtual machines, also known as domains. At least one Oracle VM Server is required, but several are needed to take advantage of clustering.
Oracle VM Server is installed on a bare metal computer, and contains the Oracle VM Agent to manage communication with Oracle VM Manager. dom0 is an abbreviation for domain zero, the management or control domain with privileged access to the hardware and device drivers. DomU is an unprivileged domain with no direct access to the hardware or device drivers. A user-domain (domU) is started and managed on an Oracle VM Server by dom0.
On x86-based systems, Oracle VM Server is based upon an updated version of the underlying Xen hypervisor technology, and includes Oracle VM Agent. It also includes a Linux kernel with support for a broad array of devices, file systems, and software RAID volume management. The Linux kernel is run as dom0 to manage one or more domU virtual machines, each of which could be Linux, Oracle Solaris, or Microsoft Windows™.
In contrast, Oracle VM Server for SPARC takes advantage of the hypervisor that is already included within the SPARC firmware, alongside the Oracle VM Agent for SPARC. The default Oracle Solaris operating system is usually promoted to act as the primary domain, which is equivalent to dom0 on x86 systems. Once the primary domain is in place, it can be used to create and manage further domains running different versions of the Oracle Solaris operating system.
Groups of Oracle VM Servers are usually clustered together to create server pools. This allows Oracle VM Manager to handle load balancing and failover for high-availability environments. Virtual machines run within a server pool and can be easily moved between the different servers that make up a server pool. Server pools also provide logical separation of servers and virtual machines. Server pools are required entities within the Oracle VM infrastructure, even if they consist of only one server.
Each Oracle VM Server maintains its own Berkeley Database, used to store local configuration and runtime information. This allows the Oracle VM Server to continue to function normally, even if Oracle VM Manager becomes unavailable for a period. Where Oracle VM Servers are clustered together, a separate cluster database, stored in the server pool file system, is shared between the servers. This allows the server pool to continue to provide clustering features, such as High Availability, even if Oracle VM Manager is unavailable.
Client Applications: Various user interfaces to Oracle VM Manager are provided, either via the graphical user interface (GUI) accessible using a web-browser; the command line interface (CLI) accessible using an SSH client; custom built applications or scripts that use the Web Services API (WS-API); or external applications, such as Oracle Enterprise Manager, or legacy utility scripts that may still make use of the legacy API over TCPS on port 54322. The legacy API is due to be deprecated in the near future and applications that are using it must be updated to use the new Web Services API instead. All communications with Oracle VM Manager are secured using either a key or certificate based technology.
Oracle VM Manager: Used to manage Oracle VM Servers, virtual machines, and resources. It is comprised of a number of subcomponents, include a web browser-based user interface; and a command line interface (CLI) allowing you to manage your infrastructure directly from the command line either via external scripts or by running manual command sequences. Each of these interfaces runs as a separate application to the Oracle VM Manager core and interfaces with this using the Web Services API.
The Oracle VM Manager core is an Oracle WebLogic Server application running on Oracle Linux. The user interface uses the Application Development Framework (ADF) application, providing a common look and feel, in line with other Oracle web-based applications. Oracle VM Manager can be on a standalone computer, or part of a virtual machine running on an instance of Oracle VM Server. While Oracle VM Manager core and the Oracle VM Manager GUI are both WebLogic applications, they are separate applications, even though they share the same process space.
Oracle VM Manager communicates with each Oracle VM Server via the Oracle VM Agent, using XML-RPC over HTTPS on port 8899. Actions on servers that are initiated within Oracle VM Manager are triggered using this method. The Oracle VM Agent on each Oracle VM Server is equally able to send notifications, statistics and event information back to Oracle VM Manager. Actions within Oracle VM Manager triggered by Oracle VM Agent are achieved using the Web Services API exposed by Oracle VM Manager and are secured using HTTPS.
While Oracle VM Manager is a critical component for configuration actions within the Oracle VM infrastructure, the virtualized environment can continue to function properly even if Oracle VM Manager experiences downtime. This includes the ability to maintain high availability and to perform live migration of virtual machines
Oracle VM Manager Database: Used by Oracle VM Manager core to store and track configuration, status changes and events. Oracle VM Manager uses a MySQL Enterprise database that is bundled in the installer and which runs on the same host where Oracle VM Manager is installed. The database is configured for the exclusive use of Oracle VM Manager and must not be used by any other applications. The database is automatically backed up on a regular schedule, and facilities are provided to perform manual backups as well.
Oracle VM Server: A managed virtualization environment providing a lightweight, secure, server platform which runs virtual machines, also known as domains. At least one Oracle VM Server is required, but several are needed to take advantage of clustering.
Oracle VM Server is installed on a bare metal computer, and contains the Oracle VM Agent to manage communication with Oracle VM Manager. dom0 is an abbreviation for domain zero, the management or control domain with privileged access to the hardware and device drivers. DomU is an unprivileged domain with no direct access to the hardware or device drivers. A user-domain (domU) is started and managed on an Oracle VM Server by dom0.
On x86-based systems, Oracle VM Server is based upon an updated version of the underlying Xen hypervisor technology, and includes Oracle VM Agent. It also includes a Linux kernel with support for a broad array of devices, file systems, and software RAID volume management. The Linux kernel is run as dom0 to manage one or more domU virtual machines, each of which could be Linux, Oracle Solaris, or Microsoft Windows™.
In contrast, Oracle VM Server for SPARC takes advantage of the hypervisor that is already included within the SPARC firmware, alongside the Oracle VM Agent for SPARC. The default Oracle Solaris operating system is usually promoted to act as the primary domain, which is equivalent to dom0 on x86 systems. Once the primary domain is in place, it can be used to create and manage further domains running different versions of the Oracle Solaris operating system.
Groups of Oracle VM Servers are usually clustered together to create server pools. This allows Oracle VM Manager to handle load balancing and failover for high-availability environments. Virtual machines run within a server pool and can be easily moved between the different servers that make up a server pool. Server pools also provide logical separation of servers and virtual machines. Server pools are required entities within the Oracle VM infrastructure, even if they consist of only one server.
Each Oracle VM Server maintains its own Berkeley Database, used to store local configuration and runtime information. This allows the Oracle VM Server to continue to function normally, even if Oracle VM Manager becomes unavailable for a period. Where Oracle VM Servers are clustered together, a separate cluster database, stored in the server pool file system, is shared between the servers. This allows the server pool to continue to provide clustering features, such as High Availability, even if Oracle VM Manager is unavailable.
Subscribe to:
Posts (Atom)