Virtualization Sysadmin

Wednesday, June 3, 2020

How to Install the Oracle Solaris OS on a Guest Domain From an Oracle Solaris ISO File

Stop and unbind the guest domain (ldg1).

primary# ldm stop ldg1
primary# ldm unbind ldg1

Add the Oracle Solaris ISO file as a secondary volume and virtual disk.
The following uses solarisdvd.iso as the Oracle Solaris ISO file, iso_vol@primary-vds0 as a secondary volume, and vdisk_iso as a virtual disk:
```
primary# ldm add-vdsdev /export/solarisdvd.iso  iso_vol@primary-vds0
primary# ldm add-vdisk vdisk_iso iso_vol@primary-vds0 ldg1
```
The following uses solarisdvd.iso as the Oracle Solaris ISO file, which is stored on the server1 server. iso_vol@primary-vds0 is a secondary volume, and vdisk_isois a virtual disk. The lofiadm command attaches a block device to the ISO image.
```
primary# lofiadm -a /net/server1/solarisdvd.iso
primary# ldm add-vdsdev /dev/lofi/1 iso_vol@primary-vds0
primary# ldm add-vdisk vdisk_iso iso_vol@primary-vds0 ldg1
```

Check to see that the Oracle Solaris ISO file is added as a secondary volume and virtual disk.

primary# ldm list-bindings
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME 
primary          active   -n-cv   SP      4     4G       0.2%  22h 45m
...
VDS 
   NAME             VOLUME         OPTIONS          DEVICE
   primary-vds0     vol1                            /dev/dsk/c2t1d0s2
   iso_vol                                          /export/solarisdvd.iso
....
------------------------------------------------------------------------------
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
ldg1             inactive -----           60    6G
...
DISK
   NAME       VOLUME               TOUT ID DEVICE  SERVER  MPGROUP
   vdisk1     vol1@primary-vds0
   vdisk_iso  iso_vol@primary-vds0
....

Bind and start the guest domain (ldg1).

primary# ldm bind ldg1
primary# ldm start ldg1
LDom ldg1 started
primary# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
 
Connecting to console "ldg1" in group "ldg1" ....
Press ~? for control options ..

Show the device aliases in the client OpenBoot PROM.

In this example, see the device aliases for vdisk_iso, which is the Oracle Solaris ISO image, and vdisk_install, which is the disk space.

ok devalias
vdisk_iso       /virtual-devices@100/channel-devices@200/disk@1
vdisk1          /virtual-devices@100/channel-devices@200/disk@0
vnet1           /virtual-devices@100/channel-devices@200/network@0
virtual-console /virtual-devices/console@1
name            aliases

On the guest domain's console, boot from vdisk_iso (disk@1) on slice f.

ok boot vdisk_iso:f
Boot device: /virtual-devices@100/channel-devices@200/disk@1:f  File and args: -s
SunOS Release 5.10 Version Generic_139555-08 64-bit
Copyright (c) 1983-2010, Oracle and/or its affiliates.  All rights reserved.

Continue with the Oracle Solaris OS installation menu.

How to Install the Oracle Solaris OS on a Guest Domain From a DVD

Insert the Oracle Solaris 10 OS or Oracle Solaris 11 OS DVD into the DVD drive.
Stop the volume management daemon, vold(1M), on the primary domain.
```
primary# svcadm disable volfs
```

Stop and unbind the guest domain (ldg1).

primary# ldm stop ldg1
primary# ldm unbind ldg1

Add the DVD with the DVD-ROM media as a secondary volume and virtual disk.
The following uses c0t0d0s2 as the DVD drive in which the Oracle Solaris media resides, dvd_vol@primary-vds0 as a secondary volume, and vdisk_cd_media as a virtual disk.
```
primary# ldm add-vdsdev options=ro /dev/dsk/c0t0d0s2 dvd_vol@primary-vds0
primary# ldm add-vdisk vdisk_cd_media dvd_vol@primary-vds0 ldg1
```

Check to see that the DVD is added as a secondary volume and virtual disk.

primary# ldm list-bindings
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME 
primary          active   -n-cv   SP      4     4G       0.2%  22h 45m
...
VDS 
   NAME             VOLUME         OPTIONS          DEVICE
   primary-vds0     vol1                            /dev/dsk/c2t1d0s2
   dvd_vol                                          /dev/dsk/c0t0d0s2
....
------------------------------------------------------------------------------
NAME             STATE    FLAGS   CONS    VCPU  MEMORY   UTIL  UPTIME
ldg1             inactive -----           60    6G
...
DISK
   NAME             VOLUME                      TOUT DEVICE  SERVER
   vdisk1           vol1@primary-vds0
   vdisk_cd_media   dvd_vol@primary-vds0
....

Bind and start the guest domain (ldg1).

primary# ldm bind ldg1
primary# ldm start ldg1
LDom ldg1 started
primary# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
 
Connecting to console "ldg1" in group "ldg1" ....
Press ~? for control options ..

Show the device aliases in the client OpenBoot PROM.

In this example, see the device aliases for vdisk_cd_media, which is the Oracle Solaris DVD, and vdisk1, which is a virtual disk on which you can install the Oracle Solaris OS.

ok devalias
vdisk_cd_media  /virtual-devices@100/channel-devices@200/disk@1
vdisk1          /virtual-devices@100/channel-devices@200/disk@0
vnet1           /virtual-devices@100/channel-devices@200/network@0
virtual-console /virtual-devices/console@1
name            aliases

On the guest domain's console, boot from vdisk_cd_media (disk@1) on slice f.

ok boot vdisk_cd_media:f
Boot device: /virtual-devices@100/channel-devices@200/disk@1:f  File and args: -s
SunOS Release 5.10 Version Generic_139555-08 64-bit
Copyright (c), 1983-2010, Oracle and/or its affiliates.  All rights reserved.

Continue with the Oracle Solaris OS installation menu.

Creating and Starting a Guest Domain

The guest domain must run an operating system that understands both the sun4v platform and the virtual devices presented by the hypervisor. Currently, this means that you must run at least the Oracle Solaris 10 11/06 OS. Running the Oracle Solaris 10 8/11 OS provides you with all the Oracle VM Server for SPARC 3.0 features. See the Oracle VM Server for SPARC 3.0 Release Notes for any specific patches that might be necessary. Once you have created default services and reallocated resources from the control domain, you can create and start a guest domain.

How to Create and Start a Guest Domain

Create a logical domain.
For example, the following command would create a guest domain named ldg1.
```
primary# ldm add-domain ldg1
```
Add CPUs to the guest domain.
For example, the following command would add eight virtual CPUs to guest domain ldg1.
```
primary# ldm add-vcpu 8 ldg1
```
Add memory to the guest domain.
For example, the following command would add 2 gigabytes of memory to guest domain ldg1.
```
primary# ldm add-memory 2G ldg1
```
Add a virtual network device to the guest domain.
For example, the following command would add a virtual network device with these specifics to the guest domain ldg1.
```
primary# ldm add-vnet vnet1 primary-vsw0 ldg1
```
Where:
- vnet1 is a unique interface name to the logical domain, assigned to this virtual network device instance for reference on subsequent set-vnet or remove-vnet subcommands.
- primary-vsw0 is the name of an existing network service (virtual switch) to which to connect.
Note - Steps 5 and 6 are simplified instructions for adding a virtual disk server device (vdsdev) to the primary domain and a virtual disk (vdisk) to the guest domain. To learn how ZFS volumes and file systems can be used as virtual disks, see How to Export a ZFS Volume as a Single-Slice Disk and Using ZFS With Virtual Disks.
Specify the device to be exported by the virtual disk server as a virtual disk to the guest domain.
You can export a physical disk, disk slice, volumes, or file as a block device. The following examples show a physical disk and a file.
- Physical Disk Example. The first example adds a physical disk with these specifics.
```
primary# ldm add-vdsdev /dev/dsk/c2t1d0s2 vol1@primary-vds0
```
  Where:
  - /dev/dsk/c2t1d0s2 is the path name of the actual physical device. When adding a device, the path name must be paired with the device name.
  - vol1 is a unique name you must specify for the device being added to the virtual disk server. The volume name must be unique to this virtual disk server instance, because this name is exported by this virtual disk server to the clients for adding. When adding a device, the volume name must be paired with the path name of the actual device.
  - primary-vds0 is the name of the virtual disk server to which to add this device.
- File Example. This second example is exporting a file as a block device.
```
primary# ldm add-vdsdev backend vol1@primary-vds0
```
  Where:
  - backend is the path name of the actual file exported as a block device. When adding a device, the back end must be paired with the device name.
  - vol1 is a unique name you must specify for the device being added to the virtual disk server. The volume name must be unique to this virtual disk server instance, because this name is exported by this virtual disk server to the clients for adding. When adding a device, the volume name must be paired with the path name of the actual device.
  - primary-vds0 is the name of the virtual disk server to which to add this device.
Add a virtual disk to the guest domain.
The following example adds a virtual disk to the guest domain ldg1.
```
primary# ldm add-vdisk vdisk1 vol1@primary-vds0 ldg1
```
Where:
- vdisk1 is the name of the virtual disk.
- vol1 is the name of the existing volume to which to connect.
- primary-vds0 is the name of the existing virtual disk server to which to connect.
Note - The virtual disks are generic block devices that are associated with different types of physical devices, volumes, or files. A virtual disk is not synonymous with a SCSI disk and, therefore, excludes the target ID in the disk label. Virtual disks in a logical domain have the following format: cNdNsN, where cN is the virtual controller, dN is the virtual disk number, and sN is the slice.
Set the auto-boot? and boot-device variables for the guest domain.
The first example command sets auto-boot? to true for guest domain ldg1.
```
primary# ldm set-var auto-boot\?=true ldg1
```
The second example command sets boot-device to vdisk1 for guest domain ldg1.
```
primary# ldm set-var boot-device=vdisk1 ldg1
```

Bind resources to the guest domain ldg1 and then list the domain to verify that it is bound.

primary# ldm bind-domain ldg1
primary# ldm list-domain ldg1
NAME          STATE    FLAGS  CONS   VCPU MEMORY   UTIL  UPTIME
ldg1          bound    -----  5000   8    2G

To find the console port of the guest domain, you can look at the output of the preceding list-domain subcommand.
You can see under the heading CONS that logical domain guest 1 (ldg1) has its console output bound to port 5000.
Connect to the console of a guest domain from another terminal by logging into the control domain and connecting directly to the console port on the local host.
```
$ ssh hostname.domain-name
$ telnet localhost 5000
```
Start the guest domain ldg1.
```
primary# ldm start-domain ldg1
```

How to Enable the Virtual Network Terminal Server Daemon

Enabling the Virtual Network Terminal Server Daemon

You must enable the virtual network terminal server daemon (vntsd) to provide access to the virtual console of each logical domain. Refer to the vntsd(1M) man page for information about how to use this daemon.

Note - Be sure that you have created the default service vconscon (vcc) on the control domain before you enable vntsd. See Creating Default Services for more information.

Use the svcadm command to enable the virtual network terminal server daemon, vntsd.
```
primary# svcadm enable vntsd
```

Use the svcs command to verify that the vntsd daemon is enabled.

primary# svcs vntsd
STATE          STIME    FMRI
online         Oct_08   svc:/ldoms/vntsd:default

Rebooting to Use Logical Domains

You must reboot the control domain for the configuration changes to take effect and for the resources to be released for other logical domains to use.

How to Reboot

Shut down and reboot the control domain.
```
primary# shutdown -y -g0 -i6
```
Note - Either a reboot or power cycle instantiates the new configuration. Only a power cycle actually boots the configuration saved to the service processor (SP), which is then reflected in the list-config output.

How to Set Up the Control Domain

Note - This procedure contains examples of resources to set for your control domain. These numbers are examples only, and the values used might not be appropriate for your control domain.

Determine whether you have cryptographic devices in the control domain.
Note that only UltraSPARC T2, UltraSPARC T2 Plus, and SPARC T3 platforms have cryptographic devices (MAUs). Because newer platforms, such as SPARC T4 systems and Fujitsu M10 systems, do not have separate cryptographic units, there is no need to assign a cryptographic accelerator on these platforms.
```
primary# ldm list -o crypto primary
```
Assign cryptographic resources to the control domain, if applicable.
The following example would assign one cryptographic resource to the control domain, primary. This leaves the remainder of the cryptographic resources available to a guest domain.
```
primary# ldm set-mau 1 primary
```
Assign virtual CPUs to the control domain.
For example, the following command would assign 8 virtual CPUs to the control domain, primary. This leaves the remainder of the virtual CPUs available to a guest domain.
```
primary# ldm set-vcpu 8 primary
```
Initiate a delayed reconfiguration on the control domain.
```
primary# ldm start-reconf primary
```
Assign memory to the control domain.
For example, the following command would assign 4 gigabytes of memory to the control domain, primary. This leaves the remainder of the memory available to a guest domain.
```
primary# ldm set-memory 4G primary
```
Add a logical domain machine configuration to the service processor (SP).
For example, the following command would add a configuration called initial.
```
primary# ldm add-config initial
```
Verify that the configuration is ready to be used at the next reboot.
```
primary# ldm list-config
factory-default
initial [current]
```
This ldm list-config command shows that the initial configuration set will be used after you perform a power cycle.

Creating virtual device services to use in the control domain as a service domain

The following virtual device services must be created to use the control domain as a service domain and to create virtual devices for other domains:

vcc – Virtual console concentrator service
vds – Virtual disk server
vsw – Virtual switch service

How to Create Default Services

Create a virtual console concentrator (vcc) service for use by the virtual network terminal server daemon (vntsd) and as a concentrator for all logical domain consoles.
For example, the following command would add a virtual console concentrator service (primary-vcc0) with a port range from 5000 to 5100 to the control domain (primary).
```
primary# ldm add-vcc port-range=5000-5100 primary-vcc0 primary
```
Create a virtual disk server (vds) to allow importing virtual disks into a logical domain.
For example, the following command adds a virtual disk server (primary-vds0) to the control domain (primary).
```
primary# ldm add-vds primary-vds0 primary
```
Create a virtual switch service (vsw) to enable networking between virtual network (vnet) devices in logical domains.
Assign a GLDv3-compliant network adapter to the virtual switch if each logical domain must communicate outside the box through the virtual switch.
- In Oracle Solaris 10, add a virtual switch service on network adapter driver to the control domain.
```
primary# ldm add-vsw net-dev=net-driver vsw-service primary
```
  For example, the following command adds a virtual switch service (primary-vsw0) on network adapter driver nxge0 to the control domain (primary):
```
primary# ldm add-vsw net-dev=nxge0 primary-vsw0 primary
```
- In Oracle Solaris 11, add a virtual switch service (primary-vsw0) on network adapter driver net0 to the control domain (primary):
```
primary# ldm add-vsw net-dev=net-driver vsw-service primary
```
  For example, the following command adds a virtual switch service (primary-vsw0) on network adapter driver net0 to the control domain (primary):
```
primary# ldm add-vsw net-dev=net0 primary-vsw0 primary
```
- The following applies to the Oracle Solaris 10 OS only and should not be performed on an Oracle Solaris 11 system.
  
  This command automatically allocates a MAC address to the virtual switch. You can specify your own MAC address as an option to the ldm add-vsw command. However, in that case, it is your responsibility to ensure that the MAC address specified does not conflict with an already existing MAC address.
  
  If the virtual switch being added replaces the underlying physical adapter as the primary network interface, it must be assigned the MAC address of the physical adapter, so that the Dynamic Host Configuration Protocol (DHCP) server assigns the domain the same IP address. See Enabling Networking Between the Control/Service Domain and Other Domains.
```
primary# ldm add-vsw mac-addr=2:04:4f:fb:9f:0d net-dev=nxge0 primary-vsw0 primary
```

Verify the services have been created by using the list-services subcommand.

Your output should look similar to the following:

primary# ldm list-services primary
VDS
    NAME             VOLUME         OPTIONS          DEVICE
    primary-vds0
 
VCC
    NAME             PORT-RANGE
    primary-vcc0     5000-5100
 
VSW
    NAME             MAC               NET-DEV   DEVICE     MODE
    primary-vsw0     02:04:4f:fb:9f:0d nxge0     switch@0   prog,promisc