Openstack Essex Deploy by Puppet on Ubuntu 12.04 HOWTO

Submitted by hswong3i on Tue, 2012-06-19 16:35

This HOWTO will guide you though a multi-node Openstack Essex deployment with Puppet on Ubuntu 12.04.

Before start I will assume you have a clean Ubuntu Server 12.04 installed with minimal packages requirement. It is strongly recommend to install Openstack for a new host, as it will modify a lot of default settings; from the other point of view, don't install Openstack on top of an online production that already well configured ;-)

Prerequisites

This environment will include 3 hosts:

1 master/proxy/controller + compute node (named controller in the following)
- vms1.hkstp.internal (eth0: 172.24.0.11/16, eth1: null)
2 compute only nodes
- vms2.hkstp.internal (eth0: 172.24.0.12/16, eth1: null)
- vms3.hkstp.internal (eth0: 172.24.0.13/16, eth1: null)

My overall design (for sure, just for internal development and testing):

Management subnet: 172.24.0.0/16 (eth0)
Floating range: 172.24.1.0/24 (eth0)
Fixed range: 10.1.0.0/16 (eth1)
Controller node address: 172.24.0.11
Default username: openstack
Default password: openstack
Default token: bdbb8df712625fa7d1e0ff1e049e8aab

Network setup example for /etc/network/interfaces (update with your dns-* accordingly):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 172.24.0.11
        netmask 255.255.0.0
        network 172.24.0.0
        broadcast 172.24.255.255
        gateway 172.24.0.1
        dns-nameservers 202.130.97.65 202.130.97.66
        dns-search hkstp.internal

auto eth1
iface eth1 inet manual
        up ifconfig eth1 up

You may also need to map above hostname statically by editing /etc/hosts as below:

127.0.0.1      localhost
172.24.0.11    vms1.hkstp.internal    vms1
172.24.0.12    vms2.hkstp.internal    vms2
172.24.0.13    vms3.hkstp.internal    vms3

Every node that is configured to be a nova volume service must have a volume group called nova-volumes.

NOTE: If you are going to use live migration functionality, pre-create system user/group id so they can map directly in cluster setup:

addgroup --system --gid 999 kvm
addgroup --system --gid 998 libvirtd
addgroup --system --gid 997 nova
adduser --system --home /var/lib/libvirt --shell /bin/false --uid 999 --gid 999 --disabled-password libvirt-qemu
adduser --system --home /var/lib/libvirt/dnsmasq --shell /bin/false --uid 998 --gid 998 --disabled-password libvirt-dnsmasq
adduser --system --home /var/lib/nova --shell /bin/false --uid 997 --gid 997 --disabled-password nova
adduser nova libvirtd

Install Puppet

(All nodes) Install puppet agent:

aptitude -y install puppet augeas-tools

(Controller node only) Install puppetmaster by APT, and also install puppetlabs_spec_helper by Gem:

aptitude -y install puppetmaster sqlite3 libsqlite3-ruby libactiverecord-ruby git rake
gem install puppetlabs_spec_helper

(All nodes) Enable pluginsync and configure the hostname of the puppetmaster:

augtool << EOF
set /files/etc/puppet/puppet.conf/agent/pluginsync true
set /files/etc/puppet/puppet.conf/agent/server vms1.hkstp.internal
save
EOF

(Controller node only) Enable storedconfig and configure database:

augtool << EOF
set /files/etc/puppet/puppet.conf/master/storeconfigs true
set /files/etc/puppet/puppet.conf/master/dbadapter sqlite3
set /files/etc/puppet/puppet.conf/master/dblocation /var/lib/puppet/server_data/storeconfigs.sqlite
save
EOF

(Controller node only) Create a dummy site manifest:

cat > /etc/puppet/manifests/site.pp << EOF
node default {
  notify { "Hey ! It works !": }
}
EOF

(Controller node only) Restart puppetmaster

/etc/init.d/puppetmaster restart

Test the puppet agents

(All nodes) Register each client with the puppetmaster:

puppet agent -vt --waitforcert 60

(Controller node only) While the puppet agent is waiting, sign the client certificates:

puppetca sign -a

There should be no error and you should see similar message as below on client:

info: Caching catalog for vms3.hkstp.internal
info: Applying configuration version '1340077073'
notice: Hey ! It works !

Install the Openstack modules for Puppet

Before keep on going it is strongly recommend to reboot your system:

reboot

(Controller node only) Install the latest revision of the modules from GIT:

cd /etc/puppet/modules
git clone git://github.com/puppetlabs/puppetlabs-openstack openstack
cd openstack
rake modules:clone

Now your /etc/puppet/modules should looks like below:

root@vms1:/etc/puppet/modules# ls -la /etc/puppet/modules/
total 80
drwxr-xr-x 20 root root 4096 Jun 19 11:55 .
drwxr-xr-x  6 root root 4096 Jun 19 11:46 ..
drwxr-xr-x  7 root root 4096 Jun 19 11:55 apt
drwxr-xr-x  7 root root 4096 Jun 19 11:54 concat
drwxr-xr-x  5 root root 4096 Jun 19 11:55 git
drwxr-xr-x  9 root root 4096 Jun 19 11:55 glance
drwxr-xr-x  6 root root 4096 Jun 19 11:55 horizon
drwxr-xr-x  9 root root 4096 Jun 19 11:55 keystone
drwxr-xr-x  7 root root 4096 Jun 19 11:54 memcached
drwxr-xr-x  9 root root 4096 Jun 19 11:55 mysql
drwxr-xr-x 11 root root 4096 Jun 19 11:55 nova
drwxr-xr-x  7 root root 4096 Jun 19 11:54 openstack
drwxr-xr-x  9 root root 4096 Jun 19 11:55 rabbitmq
drwxr-xr-x  8 root root 4096 Jun 19 11:55 rsync
drwxr-xr-x  7 root root 4096 Jun 19 11:55 ssh
drwxr-xr-x  7 root root 4096 Jun 19 11:55 stdlib
drwxr-xr-x 10 root root 4096 Jun 19 11:55 swift
drwxr-xr-x  5 root root 4096 Jun 19 11:55 sysctl
drwxr-xr-x  6 root root 4096 Jun 19 11:55 vcsrepo
drwxr-xr-x  8 root root 4096 Jun 19 11:55 xinetd

Deploy Openstack controller node on multi-node environment

(Controller node only) Some patch to latest GIT so suit for my usecase (therefore you should futher more override them with your case):

cat > /tmp/puppetlabs-openstack.patch << EOF
diff --git examples/site.pp examples/site.pp
index 879d8fa..fd38d4e 100644
--- examples/site.pp
+++ examples/site.pp
@@ -4,7 +4,9 @@
 #
 
 # deploy a script that can be used to test nova
-class { 'openstack::test_file': }
+class { 'openstack::test_file':
+  image_type => 'ubuntu',
+}
 
 ####### shared variables ##################
 
@@ -21,17 +23,17 @@ \$public_interface        = 'eth0'
 \$private_interface       = 'eth1'
 # credentials
 \$admin_email             = 'root@localhost'
-\$admin_password          = 'keystone_admin'
-\$keystone_db_password    = 'keystone_db_pass'
-\$keystone_admin_token    = 'keystone_admin_token'
-\$nova_db_password        = 'nova_pass'
-\$nova_user_password      = 'nova_pass'
-\$glance_db_password      = 'glance_pass'
-\$glance_user_password    = 'glance_pass'
-\$rabbit_password         = 'openstack_rabbit_password'
-\$rabbit_user             = 'openstack_rabbit_user'
-\$fixed_network_range     = '10.0.0.0/24'
-\$floating_network_range  = '192.168.101.64/28'
+\$admin_password          = 'openstack'
+\$keystone_db_password    = 'openstack'
+\$keystone_admin_token    = 'bdbb8df712625fa7d1e0ff1e049e8aab'
+\$nova_db_password        = 'openstack'
+\$nova_user_password      = 'openstack'
+\$glance_db_password      = 'openstack'
+\$glance_user_password    = 'openstack'
+\$rabbit_password         = 'openstack'
+\$rabbit_user             = 'openstack'
+\$fixed_network_range     = '10.1.0.0/16'
+\$floating_network_range  = '172.24.1.0/24'
 # switch this to true to have all service log at verbose
 \$verbose                 = false
 # by default it does not enable atomatically adding floating IPs
@@ -75,7 +77,7 @@ node /openstack_all/ {
 
 # multi-node specific parameters
 
-\$controller_node_address  = '192.168.101.11'
+\$controller_node_address  = '172.24.0.11'
 
 \$controller_node_public   = \$controller_node_address
 \$controller_node_internal = \$controller_node_address
@@ -83,9 +85,9 @@ \$sql_connection         = "mysql://nova:\${nova_db_password}@\${controller_node_in
 
 node /openstack_controller/ {
 
-#  class { 'nova::volume': enabled => true }
+  class { 'nova::volume': enabled => true }
 
-#  class { 'nova::volume::iscsi': }
+  class { 'nova::volume::iscsi': }
 
   class { 'openstack::controller':
     public_address          => \$controller_node_public,
@@ -142,7 +144,7 @@ node /openstack_compute/ {
     vncproxy_host      => \$controller_node_public,
     vnc_enabled        => true,
     verbose            => \$verbose,
-    manage_volumes     => true,
+    manage_volumes     => false,
     nova_volume        => 'nova-volumes'
   }

EOF
cd /etc/puppet/modules/openstack
patch -p0 < /tmp/puppetlabs-openstack.patch

Link the module's example site.pp on the controller for production (I do so therefore able to keep trace changes with GIT):

rm -rf /etc/puppet/manifests/site.pp
ln -s /etc/puppet/modules/openstack/examples/site.pp /etc/puppet/manifests/site.pp

Once everything is configured on the controller, you can now configure the controller node by:

puppet agent -vt --waitforcert 60 --certname openstack_controller

While the puppet agent is waiting, sign the client certificates:

puppetca sign -a

Now wait and have a coffee break... Once ready, access http://172.24.0.11/ and should show Openstack Dashboard as below:

Deploy Openstack compute node on multi-node environment

Once controller get ready, configure compute nodes by:

puppet agent -vt --waitforcert 60 --certname openstack_compute_vms1
puppet agent -vt --waitforcert 60 --certname openstack_compute_vms2
puppet agent -vt --waitforcert 60 --certname openstack_compute_vms3

While the puppet agent is waiting, sign the client certificates:

puppetca sign -a

Now wait and have a coffee break...

Verify your Openstack deployment

Once you have installed Openstack with Puppet (and assuming you experience no errors), the next step is to verify the installation.

Ensure that your authentication information is in the user's environment by:

source /root/openrc

For development I would like to release firewall rules for all conntection:

nova secgroup-add-rule default tcp 1 65535 0.0.0.0/0
nova secgroup-add-rule default udp 1 65535 0.0.0.0/0
nova secgroup-add-rule default icmp -1 255 0.0.0.0/0

Verify that all of the services for nova are operational by (Ctrl + C to terminate):

watch -n1 nova-manage service list

Which should give you similar result as:

Every 1.0s: nova-manage service list                                               Tue Jun 19 15:52:12 2012

2012-06-19 15:52:12 DEBUG nova.utils [req-7eb90044-238e-4ff5-b60a-cbf7fc243b2e None None] backend <module '
nova.db.sqlalchemy.api' from '/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.pyc'> from (pid=3498)
 __get_backend /usr/lib/python2.7/dist-packages/nova/utils.py:658
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth vms1                                 nova             enabled    :-)   2012-06-19 07:52:05
nova-scheduler   vms1                                 nova             enabled    :-)   2012-06-19 07:52:05
nova-cert        vms1                                 nova             enabled    :-)   2012-06-19 07:52:05
nova-compute     vms1                                 nova             enabled    :-)   2012-06-19 07:52:09
nova-volume      vms1                                 nova             enabled    :-)   2012-06-19 07:52:05
nova-network     vms1                                 nova             enabled    :-)   2012-06-19 07:52:07
nova-network     vms3                                 nova             enabled    :-)   2012-06-19 07:52:05
nova-volume      vms3                                 nova             enabled    :-)   2012-06-19 07:52:03
nova-compute     vms3                                 nova             enabled    :-)   2012-06-19 07:52:11

Run the test script in order to import default images, add key, and start it:

cp /etc/puppet/modules/openstack/files/nova_test.sh /tmp/nova_test.sh
cd /tmp
bash ./nova_test.sh

Now access http://172.24.0.11/ and test as below:

Import your keypair
Edit default security group to allow all TCP/UDP (i.e. 1 - 65535) to 0.0.0.0/0; all ICMP (i.e. -1 - 255) to 0.0.0.0/0
Allocate IP to project
Fire up a VM, with your imported keypair
Create a volume
Attach that volume to the VM
Allocate a floating IP to a VM instance
Test remote connection with your keypair + SSH

Upgrading

(Controller node only) First of all you should MANUALLY access all /etc/puppet/modules/* GIT clone and pull with latest update... That's too complicated! Let's download my lazy git-pull-all.sh script and get it done within seconds!

wget http://edin.no-ip.com/files/git-pull-all_sh
mv git-pull-all_sh /usr/local/bin/git-pull-all.sh
chmod a+x /usr/local/bin/git-pull-all.sh
git-pull-all.sh /etc/puppet/modules

Go back to controller and redeploy with latest setup:

puppet agent -vt --waitforcert 60 --certname openstack_controller

And so for compute nodes too:

puppet agent -vt --waitforcert 60 --certname openstack_compute_vms1
puppet agent -vt --waitforcert 60 --certname openstack_compute_vms2
puppet agent -vt --waitforcert 60 --certname openstack_compute_vms3

Don't forget to reboot all of your systems ;-)

References

Tags:

OpenStack

Ubuntu

Puppet

Comments

thx,this save me a lot of

Permalink Submitted by root on Tue, 2012-08-21 17:37

thx,this save me a lot of time. But I suffer many error about Puppet

Found a similar resource:

Permalink Submitted by root on Tue, 2012-09-25 15:21

Found a similar resource: http://wiki.debian.org/OpenStackPuppetHowto

Not sure who deserves the credit hehe, seems little different also.

Thanks for the guide

User menu

Openstack Essex Deploy by Puppet on Ubuntu 12.04 HOWTO

Prerequisites

Install Puppet

Test the puppet agents

Install the Openstack modules for Puppet

Deploy Openstack controller node on multi-node environment

Deploy Openstack compute node on multi-node environment

Verify your Openstack deployment

Upgrading

References

Comments

thx,this save me a lot of

Found a similar resource:

Add new comment

Plain text

Main menu

Recent blog posts

Recent comments

Who's online

User menu

You are here

Openstack Essex Deploy by Puppet on Ubuntu 12.04 HOWTO

Prerequisites

Install Puppet

Test the puppet agents

Install the Openstack modules for Puppet

Deploy Openstack controller node on multi-node environment

Deploy Openstack compute node on multi-node environment

Verify your Openstack deployment

Upgrading

References

Comments

thx,this save me a lot of

Found a similar resource:

Add new comment

Plain text

Main menu

Search form

Recent blog posts

Recent comments

Who's online