Openstack Folsom Deploy by Puppet on Ubuntu 12.04 HOWTO

Submitted by hswong3i on Wed, 2012-11-07 13:57

This HOWTO will guide you though a multi-node Openstack Folsom deployment with Puppet on Ubuntu 12.04.

Before start I will assume you have a clean Ubuntu Server 12.04 installed with minimal packages requirement. It is strongly recommend to install Openstack for a new host, as it will modify a lot of default settings; from the other point of view, don't install Openstack on top of an online production that already well configured ;-)

Prerequisites

This environment will include 3 hosts:

1 master/proxy/controller
- vms1.hkstp.internal (eth0: 172.24.0.11/16, eth1: null)
2 compute only nodes
- vms2.hkstp.internal (eth0: 172.24.0.12/16, eth1: null)
- vms3.hkstp.internal (eth0: 172.24.0.13/16, eth1: null)

My overall design (for sure, just for internal development and testing):

Management subnet: 172.24.0.0/16 (eth0)
Floating range: 172.24.1.0/24 (eth0)
Fixed range: 10.1.0.0/16 (eth1)
Controller node address: 172.24.0.11
Default username: admin
Default password: openstack
Default token: bdbb8df712625fa7d1e0ff1e049e8aab

Network setup example for /etc/network/interfaces (update with your dns-* accordingly):

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
        address 172.24.0.11
        netmask 255.255.0.0
        network 172.24.0.0
        broadcast 172.24.255.255
        gateway 172.24.0.1
        dns-nameservers 202.130.97.65 202.130.97.66
        dns-search hkstp.internal

auto eth1
iface eth1 inet manual
        up ifconfig $IFACE 0.0.0.0 up
        up ifconfig $IFACE promisc

You may also need to map above hostname statically by editing /etc/hosts as below:

127.0.0.1       localhost
172.24.0.11     vms1.hkstp.internal     vms1
172.24.0.12     vms2.hkstp.internal     vms2
172.24.0.13     vms3.hkstp.internal     vms3

Every node that is configured to be a nova volume service must have a volume group called nova-volumes.

NOTE: If you are going to use live migration functionality, pre-create system user/group id so they can map directly in cluster setup:

addgroup --system --gid 999 kvm
addgroup --system --gid 998 libvirtd
addgroup --system --gid 997 nova
adduser --system --home /var/lib/libvirt --shell /bin/false --uid 999 --gid 999 --disabled-password libvirt-qemu
adduser --system --home /var/lib/libvirt/dnsmasq --shell /bin/false --uid 998 --gid 998 --disabled-password libvirt-dnsmasq
adduser --system --home /var/lib/nova --shell /bin/false --uid 997 --gid 997 --disabled-password nova
adduser nova libvirtd

Using the Puppet Labs Package Repositories

We hope to use the latest version of puppet, in order to works with github checkout:

wget http://apt.puppetlabs.com/puppetlabs-release-quantal.deb
dpkg -i puppetlabs-release-quantal.deb
aptitude update

Enable Ubuntu Cloud Archive Repositories

Refer to Ubuntu CloudArchive page, execute following command to enable folsom support for Ubuntu 12.04:

cat >> /etc/apt/sources.list <<-EOF
deb http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/folsom main
EOF
aptitude install ubuntu-cloud-keyring
aptitude update

Install Puppet

(All nodes) Install puppet agent:

aptitude -y install puppet augeas-tools

(Controller node only) Install puppetmaster by APT, and also install puppetlabs_spec_helper by Gem:

aptitude -y install puppetmaster sqlite3 libsqlite3-ruby libactiverecord-ruby git rake
gem install puppetlabs_spec_helper

(All nodes) Enable pluginsync and configure the hostname of the puppetmaster:

augtool << EOF
set /files/etc/puppet/puppet.conf/agent/pluginsync true
set /files/etc/puppet/puppet.conf/agent/server vms1.hkstp.internal
save
EOF

(Controller node only) Enable storedconfig and configure database:

augtool << EOF
set /files/etc/puppet/puppet.conf/master/storeconfigs true
set /files/etc/puppet/puppet.conf/master/dbadapter sqlite3
save
EOF

(Controller node only) Create a dummy site manifest:

cat > /etc/puppet/manifests/site.pp << EOF
node default {
  notify { "Hey ! It works !": }
}

EOF

(Controller node only) Restart puppetmaster

/etc/init.d/puppetmaster restart

Test the puppet agents

(All nodes) Register each client with the puppetmaster:

puppet agent -vt --waitforcert 60

(Controller node only) While the puppet agent is waiting, sign the client certificates:

puppet cert --sign --all

There should be no error and you should see similar message as below on client:

info: Caching catalog for vms3.hkstp.internal
info: Applying configuration version '1340077073'
notice: Hey ! It works !

Install the Openstack modules for Puppet

Before keep on going it is strongly recommend to reboot your system:

reboot

(Controller node only) Install the latest revision of the modules from GIT:

cd /etc/puppet/modules
git clone --branch folsom git://github.com/puppetlabs/puppetlabs-openstack.git openstack
cd openstack
rake modules:clone

Now your /etc/puppet/modules should looks like below:

root@vms1:/etc/puppet/modules# ls -la /etc/puppet/modules/
total 100
drwxr-xr-x 25 root root 4096 Nov  6 11:06 .
drwxr-xr-x  5 root root 4096 Nov  6 11:14 ..
drwxr-xr-x  9 root root 4096 Nov  6 11:06 apache
drwxr-xr-x  7 root root 4096 Nov  6 11:06 apt
drwxr-xr-x  6 root root 4096 Nov  6 11:06 cinder
drwxr-xr-x  7 root root 4096 Nov  7 08:58 concat
drwxr-xr-x  5 root root 4096 Nov  6 11:06 git
drwxr-xr-x  9 root root 4096 Nov  6 11:06 glance
drwxr-xr-x  6 root root 4096 Nov  6 11:05 horizon
drwxr-xr-x  6 root root 4096 Nov  7 08:58 inifile
drwxr-xr-x  9 root root 4096 Nov  6 11:05 keystone
drwxr-xr-x  8 root root 4096 Nov  7 08:57 memcached
drwxr-xr-x  9 root root 4096 Nov  6 11:06 mysql
drwxr-xr-x 10 root root 4096 Nov  6 11:06 nova
drwxr-xr-x  8 root root 4096 Nov  7 09:04 openstack
drwxr-xr-x  6 root root 4096 Nov  6 11:06 quantum
drwxr-xr-x  9 root root 4096 Nov  6 11:06 rabbitmq
drwxr-xr-x  8 root root 4096 Nov  6 11:05 rsync
drwxr-xr-x  7 root root 4096 Nov  6 11:05 ssh
drwxr-xr-x  7 root root 4096 Nov  6 11:05 stdlib
drwxr-xr-x 10 root root 4096 Nov  6 11:05 swift
drwxr-xr-x  5 root root 4096 Nov  6 11:06 sysctl
drwxr-xr-x  6 root root 4096 Nov  6 11:06 vcsrepo
drwxr-xr-x  5 root root 4096 Nov  6 11:06 vswitch
drwxr-xr-x  8 root root 4096 Nov  6 11:06 xinetd

Deploy Openstack controller node on multi-node environment

(Controller node only) Some patch to latest GIT so suit for my usecase (therefore you should futher more override them with your case):

cat > /tmp/puppetlabs-openstack.patch << EOF
diff --git a/examples/site.pp b/examples/site.pp
index a648b73..a5ff73f 100644
--- a/examples/site.pp
+++ b/examples/site.pp
@@ -24,30 +24,30 @@ \$fixed_network_range     = '10.1.0.0/16'
 \$floating_network_range  = '172.24.1.0/24'
 
 # Database settings.
-\$mysql_root_password     = 'mysql_root_password'
-\$keystone_db_password    = 'keystone_db_password'
-\$glance_db_password      = 'glance_db_password'
-\$nova_db_password        = 'nova_db_password'
-\$cinder_db_password      = 'cinder_db_password'
-\$quantum_db_password     = 'quantum_db_password'
+\$mysql_root_password     = 'openstack'
+\$keystone_db_password    = 'openstack'
+\$glance_db_password      = 'openstack'
+\$nova_db_password        = 'openstack'
+\$cinder_db_password      = 'openstack'
+\$quantum_db_password     = 'openstack'
 
 # Rabbit settings.
-\$rabbit_password         = 'rabbit_password'
+\$rabbit_password         = 'openstack'
 \$rabbit_user             = 'nova'
 
 # Keystone settings.
 \$admin_email             = 'root@localhost'
-\$admin_password          = 'keystone_admin'
-\$keystone_admin_token    = 'keystone_admin_token'
-\$glance_user_password    = 'glance_user_password'
-\$nova_user_password      = 'nova_user_password'
-\$cinder_user_password    = 'cinder_user_password'
-\$quantum_user_password   = 'quantum_user_password'
+\$admin_password          = 'openstack'
+\$keystone_admin_token    = 'bdbb8df712625fa7d1e0ff1e049e8aab'
+\$glance_user_password    = 'openstack'
+\$nova_user_password      = 'openstack'
+\$cinder_user_password    = 'openstack'
+\$quantum_user_password   = 'openstack'
 
 # Misc settings.
 \$libvirt_type            = 'kvm'
 \$network_type            = 'nova'
-\$secret_key              = 'secret_key'
+\$secret_key              = 'bdbb8df712625fa7d1e0ff1e049e8aab'
 \$verbose                 = true
 
 #### End Shared Variables #################
@@ -214,7 +214,7 @@ node /openstack_controller/ {
     horizon_app_links       => undef,
     # General.
     verbose                 => \$verbose,
-    purge_nova_config       => true,
+    purge_nova_config       => false,
   }
 
   class { 'openstack::auth_file':
@@ -254,7 +254,7 @@ node /openstack_compute/ {
     vncproxy_host          => \$controller_node_public,
     vnc_enabled            => true,
     verbose                => \$verbose,
-    purge_nova_config      => true,
+    purge_nova_config      => false,
   }
 
   # TODO Not sure why this is required.
EOF
cd /etc/puppet/modules/openstack
patch -p1 < /tmp/puppetlabs-openstack.patch

Import the module's example site.pp on the controller for production:

cat >> /etc/puppet/manifests/site.pp <<-EOF
import '../modules/openstack/examples/site.pp'
EOF

Once everything is configured on the controller, you can now configure the controller node by:

puppet agent -vt --waitforcert 60 --certname openstack_controller

While the puppet agent is waiting, sign the client certificates:

puppet cert --sign --all

Now wait and have a coffee break... Once ready, access http://172.24.0.11/horizon/ and should show Openstack Dashboard as below:

Deploy Openstack compute node on multi-node environment

Once controller get ready, configure compute nodes by:

puppet agent -vt --waitforcert 60 --certname openstack_compute2
puppet agent -vt --waitforcert 60 --certname openstack_compute3

While the puppet agent is waiting, sign the client certificates:

puppet cert --sign --all

Now wait and have a coffee break...

NOTE: After install libvirt it will auto start a NAT network interface once reboot, and your nova-network setting may get into trobule, so let's disable it by:
rm -rf /etc/libvirt/qemu/networks/autostart/default.xml

Verify your Openstack deployment

Once you have installed Openstack with Puppet (and assuming you experience no errors), the next step is to verify the installation.

Ensure that your authentication information is in the user's environment by:

source /root/openrc

For development I would like to release firewall rules for all conntection:

nova --no-cache secgroup-add-rule default tcp 1 65535 0.0.0.0/0
nova --no-cache secgroup-add-rule default udp 1 65535 0.0.0.0/0
nova --no-cache secgroup-add-rule default icmp -1 255 0.0.0.0/0

I have 2 internal IP range that have public IP DMZ mapping, therefore I would like to create 2 additional floating IP pool:
nova-manage floating create --pool=primary --ip_range=172.24.145.16/28
nova-manage floating create --pool=secondary --ip_range=172.24.234.64/29

I hope to test with both 12.04 and 12.10 cloud image so import as below (using below test_nova.sh will also import 12.04 image for you):

cd /tmp
wget -c https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
glance add name='cirros-amd64' is_public=true container_format=bare disk_format=qcow2 < cirros-0.3.0-x86_64-disk.img
wget -c http://cloud-images.ubuntu.com/precise/current/precise-server-cloudimg-amd64-disk1.img
glance add name="precise-amd64" is_public=true container_format=ovf disk_format=qcow2 < precise-server-cloudimg-amd64-disk1.img
wget -c http://cloud-images.ubuntu.com/quantal/current/quantal-server-cloudimg-amd64-disk1.img
glance add name="quantal-amd64" is_public=true container_format=ovf disk_format=qcow2 < quantal-server-cloudimg-amd64-disk1.img

Verify that all of the services for nova are operational by (Ctrl + C to terminate):

watch -n1 nova-manage service list

Which should give you similar result as:

Every 1.0s: nova-manage service list                                               Wed Nov  7 09:42:06 2012

2012-11-07 09:42:07 DEBUG nova.utils [req-c6335d9d-8549-4a0e-b6e4-2983a6acaf93 None None] backend <module '
nova.db.sqlalchemy.api' from '/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.pyc'> from (pid=8170)
 __get_backend /usr/lib/python2.7/dist-packages/nova/utils.py:494
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth vms1                                 nova             enabled    :-)   2012-11-07 01:42:03
nova-scheduler   vms1                                 nova             enabled    :-)   2012-11-07 01:42:02
nova-cert        vms1                                 nova             enabled    :-)   2012-11-07 01:42:01
nova-network     vms2                                 nova             enabled    :-)   2012-11-07 01:42:05
nova-compute     vms2                                 nova             enabled    :-)   2012-11-07 01:42:03
nova-network     vms3                                 nova             enabled    :-)   2012-11-07 01:41:58
nova-compute     vms3                                 nova             enabled    :-)   2012-11-07 01:42:05

Run the test script in order to import default images, add key, and start it:

cd /tmp
bash ./nova_test.sh

Now access http://172.24.0.11/horizon/ and test as below:

Import your keypair
Edit default security group to allow all TCP/UDP (i.e. 1 - 65535) to 0.0.0.0/0; all ICMP (i.e. -1 - 255) to 0.0.0.0/0
Allocate IP to project
Fire up a VM, with your imported keypair
Create a volume
Attach that volume to the VM
Allocate a floating IP to a VM instance
Test remote connection with your keypair + SSH

Upgrading

(Controller node only) First of all you should MANUALLY access all /etc/puppet/modules/* GIT clone and pull with latest update... That's too complicated! Let's download my lazy git-pull-all.sh script and get it done within seconds!

wget http://edin.no-ip.com/files/git-pull-all_sh
mv git-pull-all_sh /usr/local/bin/git-pull-all.sh
chmod a+x /usr/local/bin/git-pull-all.sh
git-pull-all.sh /etc/puppet/modules

Go back to controller and redeploy with latest setup:

puppet agent -vt --waitforcert 60 --certname openstack_controller

And so for compute nodes too:

puppet agent -vt --waitforcert 60 --certname openstack_compute2
puppet agent -vt --waitforcert 60 --certname openstack_compute3

Don't forget to reboot all of your systems ;-)

References

Tags:

OpenStack

Ubuntu

Puppet

Comments

some problems

Permalink Submitted by CplusHUa on Mon, 2012-11-12 22:44

perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LC_PAPER = "zh_CN",
LC_ADDRESS = "zh_CN",
LC_MONETARY = "zh_CN",
LC_NUMERIC = "zh_CN",
LC_TELEPHONE = "zh_CN",
LC_IDENTIFICATION = "zh_CN",
LC_MEASUREMENT = "zh_CN",
LC_TIME = "zh_CN",
LC_NAME = "zh_CN",
LANG = "en_US.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

添加用户和组的时候会报告这个，需要执行一下
echo "export LC_ALL=C" >> /root/.bashrc
source /root/.bashrc
这样才可以～～

User menu

Openstack Folsom Deploy by Puppet on Ubuntu 12.04 HOWTO

Prerequisites

Using the Puppet Labs Package Repositories

Enable Ubuntu Cloud Archive Repositories

Install Puppet

Test the puppet agents

Install the Openstack modules for Puppet

Deploy Openstack controller node on multi-node environment

Deploy Openstack compute node on multi-node environment

Verify your Openstack deployment

Upgrading

References

Comments

some problems

Add new comment

Plain text

Main menu

Recent blog posts

Recent comments

Who's online

User menu

You are here

Openstack Folsom Deploy by Puppet on Ubuntu 12.04 HOWTO

Prerequisites

Using the Puppet Labs Package Repositories

Enable Ubuntu Cloud Archive Repositories

Install Puppet

Test the puppet agents

Install the Openstack modules for Puppet

Deploy Openstack controller node on multi-node environment

Deploy Openstack compute node on multi-node environment

Verify your Openstack deployment

Upgrading

References

Comments

some problems

Add new comment

Plain text

Main menu

Search form

Recent blog posts

Recent comments

Who's online