Monthly Archives: October 2014

Commit a VMware snapshot from ESXi as it doesn’t appear in vCenter or command line tools

First of all we get the ID of our VM:

# vim-cmd vmsvc/getallvms

Check his snapshots:

# vim-cmd vmsvc/snapshot.get <vmid>
Get Snapshot:

We can see that we don’t have any snapshot but that’s not true:

# ls -l *vmdk
-rw-------    1 root     root       6554112 Oct 30 20:55 vm-000001-ctk.vmdk
-rw-------    1 root     root   16559525888 Oct 30 20:55 vm-000001-delta.vmdk
-rw-------    1 root     root           393 Apr 21  2014 vm-000001.vmdk
-rw-------    1 root     root       6554112 Apr  3  2014 vm-ctk.vmdk
-rw-------    1 root     root  214748364800 Apr  3  2014 vm-flat.vmdk
-rw-------    1 root     root           585 Apr  2  2014 vm.vmdk

In order to fix it, first of all shutdown the VM as it can’t be fixed online:

# vim-cmd vmsvc/power.shutdown <vmid>

And we commit the snapshot manually from command line to a new disk (it can take some time):

# vmkfstools -i vm-000001.vmdk new.vmdk
Destination disk format: VMFS zeroedthick
Cloning disk 'vm-000001.vmdk'...
Clone: 100% done.

If the task was successfully now we overwrite our old files and delete the delta disk (I suppose that you have backups just in case something goes south):

# mv new-ctk.vmdk vm-ctk.vmdk
# mv new-flat.vmdk vm-flat.vmdk
# mv new.vmdk vm.vmdk
# rm *000001*

And finally edit your vmx/vmdk file to reflect this changes:

# grep vmdk *vmx
scsi0:0.fileName = "vm.vmdk"
# grep vmdk vm.vmdk 
RW 419430400 VMFS "vm-flat.vmdk"
changeTrackPath="vm-ctk.vmdk"

And if you powerup your VM everything should be back to normal!

If you have more than one snapshot this might be handy:

* http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1004545

Fix foreman reports in Satellite 6.0

Foreman reports in Satellite 6.0 doesn’t work out of the box, first we need to setup our SMTP settings.

Error that we face without SMTP configured:

[email protected]:/var/log/foreman# /usr/sbin/foreman-rake reports:summarize
rake aborted!
getaddrinfo: Name or service not known

Tasks: TOP => reports:summarize
(See full trace by running task with --trace)

We just need to configure our settings in /etc/foreman/email.yaml like:

[email protected]:/var/log/foreman# cat /etc/foreman/email.yaml
# Outgoing email settings
production:
  delivery_method: :smtp
  smtp_settings:
    address: smtp.example.com
    port: 25
    domain: example.com
    authentication: :none

And if we run our reports once again we’ll get our email

Upgrading from a new host through Satellite 6.0 returns 403 Forbidden error

By default installation of Satellite 6.0 doesn’t update python-rhsm, if we don’t update we’ll hit this bug during a yum update:

* https://bugzilla.redhat.com/show_bug.cgi?id=1112833

Example:

https://rhs6/pulp/repos/fite_labs/DEV/rhel-6-server-x86_64-cv/content/dist/rhel/server/6/6Server/x86_64/os/repodata/repomd.xml: [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 Forbidden"
Trying other mirror.
Error: Cannot retrieve repository metadata (repomd.xml) for repository: rhel-6-server-rpms. Please verify its path and try again

Easiest solution is to upgrade python-rhsm package and reboot the server (or restart katello service).

[email protected]:~# yum upgrade
Loaded plugins: product-id, subscription-manager
Resolving Dependencies
--> Running transaction check
---> Package python-rhsm.x86_64 0:1.10.12-2.el7 will be updated
---> Package python-rhsm.x86_64 0:1.12.5-1.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================================================
 Package                                  Arch                                Version                                      Repository                                  Size
============================================================================================================================================================================
Updating:
 python-rhsm                              x86_64                              1.12.5-1.el7                                 katello-local                              112 k

Transaction Summary
============================================================================================================================================================================
Upgrade  1 Package

Total download size: 112 k
Is this ok [y/d/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : python-rhsm-1.12.5-1.el7.x86_64                                                                                                                          1/2 
  Cleanup    : python-rhsm-1.10.12-2.el7.x86_64                                                                                                                         2/2 
  Verifying  : python-rhsm-1.12.5-1.el7.x86_64                                                                                                                          1/2 
  Verifying  : python-rhsm-1.10.12-2.el7.x86_64                                                                                                                         2/2 

Updated:
  python-rhsm.x86_64 0:1.12.5-1.el7                                                                                                                                         
Complete!
[email protected]:~# reboot

And try again to update from a new host and that it would work! Magic!

First steps with Hammer and Red Hat Satellite 6.0

Instead of using the web interface Red Hat Satellite 6.0 offers a neat CLI utility called hammer.

In order to use it, first of all we set our default config:

# mkdir ~/.hammer
# chmod 600 ~/.hammer
# cat  ~/.hammer/cli_config.yml
   :foreman:
       :host: 'https://xxx'
       :username: 'admin'
       :password: 'xxx'

And then we’re ready to hammer!

Create a new organization and user admin into it:

# hammer organization create --name=fite.cat.lab --label=fite.cat.lab
# hammer organization add-user --user=admin --name=fite.cat.lab

And a new location:

# hammer location create --name=Catalonia
# hammer location add-user --name=Catalonia --user=admin
# hammer location add-organization --name=Catalonia --organization=fite.cat.lab

Create a domain:

# hammer domain create --name='fite.cat.lab'

Create a subnet:

# hammer subnet create --domain-ids=1 --gateway='10.1.1.1' --mask='255.255.255.0' --name='10.1.1.0/24'  --tftp-id=1 --network='10.1.1.0' --dns-primary='10.1.1.1'

Associate domain/subnet to our organization/location through the web portal:

# hammer organization add-subnet --subnet-id=1 --name='fite.cat.lab'
# hammer organization add-domain --domain-id=1 --name='fite.cat.lab'

Upload our manifest.zip (created in RH Portal) to our org and list our products:

# hammer subscription upload --file manifest.zip  --organization=fite.cat.lab
# hammer product list --organization fite.cat.lab
   ---|---------------------------------|-------------|--------------|--------------|-----------
   ID | NAME                            | DESCRIPTION | ORGANIZATION | REPOSITORIES | SYNC STATE
   ---|---------------------------------|-------------|--------------|--------------|-----------
   8  | Oracle Java for RHEL Server     |             | fite.cat.lab | 0            | not_synced
   9  | Red Hat Enterprise Linux Server |             | fite.cat.lab | 0            | not_synced
   7  | Red Hat Beta                    |             | fite.cat.lab | 0            | not_synced
   ---|---------------------------------|-------------|--------------|--------------|-----------

List all repositories included in a previous imported product:

# hammer repository-set list --organization=fite.cat.lab --product 'Red Hat Enterprise Linux Server'
   -----|-----------|------------------------------------------------------------------------------------------
   ID   | TYPE      | NAME                                                                                     
   -----|-----------|------------------------------------------------------------------------------------------
   861  | file      | Red Hat Enterprise Linux 5 Server Beta (Source ISOs)                                     
   3535 | yum       | Red Hat Enterprise Linux 5 Server Beta - Optional Productivity Applications (Debug RPMs) 
   ...

Enable repositories that we need, in my case all RHEL7 related ones:

# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - Fastrack (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - Optional (RPMs)'  
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - Extras (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - RH Common (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - Optional Fastrack (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - Supplementary (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'RHN Tools for Red Hat Enterprise Linux 7 Server (RPMs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server (ISOs)'
# hammer repository-set enable --organization fite.cat.lab --product 'Red Hat Enterprise Linux Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server (Kickstart)'
# hammer repository-set enable --organization fite.cat.lab --product 'Oracle Java for RHEL Server' --basearch='x86_64' --releasever='7Server' --name 'Red Hat Enterprise Linux 7 Server - Oracle Java (RPMs)'

Then we create a new product for Puppet modules in forge:

# hammer product create --name='Forge' --organization=fite.cat.lab
# hammer repository create --name='Puppet Forge' --organization=fite.cat.lab --product='Forge' --content-type='puppet' --publish-via-http=true --url=https://forge.puppetlabs.com

And finally we create a new product for EPEL packages:

# hammer product create --name='EPEL' --organization=fite.cat.lab
# hammer repository create --name='EPEL 7 - x86_64' --organization=fite.cat.lab --product='EPEL' --content-type='yum' --publish-via-http=true --url=http://dl.fedoraproject.org/pub/epel/7/x86_64/

Then we can sync all repositories that we've enabled with this simple script:

# for i in $(hammer --csv repository list --organization=fite.cat.lab  | awk -F, {'print $1'} | grep -vi '^ID'); do hammer repository synchronize --id ${i} --organization=fite.cat.lab --async; done

Create 3 environments DEV->PRE->PROD:

# hammer lifecycle-environment create --name='DEV' --prior='Library' --organization=fite.cat.lab
# hammer lifecycle-environment create --name='PRE' --prior='DEV' --organization=fite.cat.lab
# hammer lifecycle-environment create --name='PRO' --prior='PRE' --organization=fite.cat.lab

Create a daily sync plan:

# hammer sync-plan create --interval=daily --name='Daily sync' --organization=fite.cat.lab
# hammer sync-plan list --organization=fite.cat.lab

And associate this plan to our products, it must be done by sync-plan-id, not name otherwise hammer doesn't work:

# hammer product set-sync-plan --sync-plan-id=4 --organization=fite.cat.lab --name='Oracle Java for RHEL Server'
# hammer product set-sync-plan --sync-plan-id=4 --organization=fite.cat.lab --name='Red Hat Enterprise Linux Server'
# hammer product set-sync-plan --sync-plan-id=4 --organization=fite.cat.lab --name='Forge'
# hammer product set-sync-plan --sync-plan-id=4 --organization=fite.cat.lab --name='EPEL'

Create a content view for RHEL 7 server x86_64:

# hammer content-view create --name='rhel-7-server-x86_64-cv' --organization=fite.cat.lab
# for i in $(hammer --csv repository list --organization=fite.cat.lab | awk -F, {'print $1'} | grep -vi '^ID'); do hammer content-view add-repository --name='rhel-7-server-x86_64-cv' --organization=fite.cat.lab --repository-id=${i}; done

Publish this content view to Library:

# hammer content-view publish --name="rhel-7-server-x86_64-cv" --organization=fite.cat.lab --async

Promote this content view to DEV,PRE,PRO (we need to use the IDs or hammer doesn't like it, so let's get them):

# hammer content-view list --organization=fite.cat.lab 
   ----------------|---------------------------|---------------------------|-----------|---------------------------------------
   CONTENT VIEW ID | NAME                      | LABEL                     | COMPOSITE | REPOSITORY IDS                        
   ----------------|---------------------------|---------------------------|-----------|---------------------------------------
   3               | rhel-7-server-x86_64-cv   | rhel-7-server-x86_64-cv   |           | 33, 25, 31, 27, 24, 34, 26, 23, 28, 29
  ...
# hammer lifecycle-environment list --organization=fite.cat.lab
   ---|---------|--------
   ID | NAME    | PRIOR  
   ---|---------|--------
   2  | Library |        
   5  | PRO     | PRE    
   4  | PRE     | DEV    
   3  | DEV     | Library
   ---|---------|--------

# hammer content-view version promote --organization=fite.cat.lab --lifecycle-environment=3 --id=3 --async
# hammer content-view version promote --organization=fite.cat.lab --lifecycle-environment=4 --id=3 --async
# hammer content-view version promote --organization=fite.cat.lab --lifecycle-environment=5 --id=3 --async

Create a host collection for RHEL7:

# hammer host-collection create --name='RHEL 7 x86_64' --organization=fite.cat.lab

Create an activation key for our environments:

# hammer activation-key create --name='rhel-7-server-x86_64-ak-dev' --organization=fite.cat.lab --content-view='rhel-7-server-x86_64-cv' --lifecycle-environment='DEV'
# hammer activation-key create --name='rhel-7-server-x86_64-ak-pre' --organization=fite.cat.lab --content-view='rhel-7-server-x86_64-cv' --lifecycle-environment='PRE'
# hammer activation-key create --name='rhel-7-server-x86_64-ak-pro' --organization=fite.cat.lab --content-view='rhel-7-server-x86_64-cv' --lifecycle-environment='PRO'

Associate each activation key to our host collection:

# hammer activation-key add-host-collection --name='rhel-7-server-x86_64-ak-dev' --host-collection='RHEL 7 x86_64' --organization=fite.cat.lab
# hammer activation-key add-host-collection --name='rhel-7-server-x86_64-ak-pre' --host-collection='RHEL 7 x86_64' --organization=fite.cat.lab
# hammer activation-key add-host-collection --name='rhel-7-server-x86_64-ak-pro' --host-collection='RHEL 7 x86_64' --organization=fite.cat.lab

And we add all subscriptions that we have available to our keys:

   # for i in $(hammer --csv activation-key list --organization=fite.cat.lab | awk -F, {'print $1'} | grep -vi '^ID'); do for j in $(hammer --csv subscription list --organization=fite.cat.lab  | awk -F, {'print $8'} | grep -vi '^ID'); do hammer   activation-key add-subscription --id ${i} --subscription-id ${j}; done; done

Associate a partition table to OS:

# PARTID=$(hammer --csv partition-table list | grep 'Kickstart default' | awk -F, {'print $1'})
# for i in $(hammer --csv os list | awk -F, {'print $1'} | grep -vi '^ID')
  do
    hammer partition-table add-operatingsystem --id="${PARTID}" --operatingsystem-id="${i}"  
  done

Associate kickstart PXE template to OS:

# PXEID=$(hammer --csv template list | grep 'Kickstart default PXELinux' | awk -F, {'print $1'})
# SATID=$(hammer --csv template list | grep 'Satellite Kickstart Default' | awk -F, {'print $1'})
# for i in $(hammer --csv os list | awk -F, {'print $1'} | grep -vi '^ID')
  do
    hammer template add-operatingsystem --id="${PXEID}" --operatingsystem-id="${i}"
    hammer os set-default-template --id="${i}" --config-template-id="${PXEID}"
    hammer os add-config-template --id="${i}" --config-template-id="${SATID}"
    hammer os set-default-template --id="${i}" --config-template-id="${SATID}"
   done

And we create a RHEL7 hostgroup:

# MEDID=$(hammer --csv medium list | grep 'Red_Hat_7_Server_Kickstart_x86_64_7Server' | awk -F, {'print $1'})
# ENVID=$(hammer --csv environment list | grep rhel_7_server_x86_64  | grep -i dev | grep -v epel | awk -F, {'print $1'})
# PARTID=$(hammer --csv partition-table list | grep 'Kickstart default' | awk -F, {'print $1'})
# OSID=$(hammer --csv os list | grep 'RedHat 7.0' | awk -F, {'print $1'})
# CAID=1
# PROXYID=1
# hammer hostgroup create --architecture="x86_64" --domain="${DOM}" --environment-id="${ENVID}" --medium-id="${MEDID}" --name="${HGRHEL6DEV}" --subnet="${NETNAME}" --ptable-id="${PARTID}" --operatingsystem-id="${OSID}" --puppet-ca-proxy-id="${CAID}" --puppet-proxy-id="${PROXYID}"

And finally creating a new host via PXE should work...or not 🙂

Later on I'll try to post a complete script via GitHub as copy/pasting this is a nightmare, stay tuned!

Using pulp client in Satellite 6.0 from command line

In order to use pulp client we need to install it and configure it:

# yum install -y pulp-admin-client
# mkdir ~/.pulp/
# vi ~/.pulp/admin.conf
[server]
verify_ssl = false
host = <FQDN Satellite 6 server>

Then we need our pulp admin password:

# grep -i ^default_password /etc/pulp/server.conf 
default_password: <string>

And that’s it, we’re ready to use pulp client using admin user:

# pulp-admin login -u admin
Enter password: 
Successfully logged in. Session certificate will expire at Oct  2 22:53:05 2014 GMT.

Some example listing all our tasks:

# pulp-admin  tasks list
+----------------------------------------------------------------------+
                                 Tasks
+----------------------------------------------------------------------+

Operations:  publish
Resources:   test_labs-EPEL-EPEL_7_-_x86_64 (repository)
State:       Successful
Start Time:  2014-09-25T21:15:21Z
Finish Time: 2014-09-25T21:15:21Z
Task Id:     c281fcac-61f5-4679-835e-72ded469c26c
...