 _____ ______ ______  _____       _             _              _____ 
/  ___||  _  \|  _  \/  __ \     | |           | |            |  _  |          Developed By
\ `--. | | | || | | || /  \/     | |      __ _ | |__   __   __ \ V /    --------------------------
 `--. \| | | || | | || |         | |     / _` || '_ \  \ \ / / / _ \    Rutger Blom  &  Luis Chanu
/\__/ /| |/ / | |/ / | \__/\  _  | |____| (_| || |_) |  \ V / | |_| |   NSX vExpert     VCDX #246
\____/ |___/  |___/   \____/ (_) \_____/ \__,_||_.__/    \_/  \_____/

Description
Requirements
- Recommendations
Preparations
Tested Ansible Module Versions
Upgrade Considerations
Networking
IP Address Assignments
- Underlay Network
- Overlay Network
SDDC.Lab Default Credentials
Usage
Project Features
Known Items
Issues With Various Software Versions
More Information
Credits

Description

This repository contains Ansible scripts that perform fully automated deployments of complete nested VMware SDDC Pods. The primary use case is consistent and speedy provisioning of nested VMware SDDC lab environments.

Each Pod by default contains:

A VyOS Router
vCenter Server
ESXi Hosts
vSAN Storage
NSX Local Manager
NSX Edge Nodes
Aria Operations for Logs

Additional products and solutions can be enabled for automated deployment and configuration within a Pod:

A DNS/NTP Server (multi-Pod)
Avi Load Balancer
vSphere Kubernetes Service
NSX Federation
VMware Cloud Foundation

Physicaloverview

Requirements

The following are the requirements for successful Pod deployments:

A physical ESXi host running version 6.7 or higher.
A virtual machine with a modern version of Ubuntu (used as the Ansible controller)
The default deployment settings require DNS name resolution. You can leverage an existing DNS server, but it must be configured with the required forward and reverse zones and support dynamic updates.
Access to VMware product installation media.
For deploying NSX you will need an NSX license (Check out VMUG Advantage
If IPv6 deployment is enabled (Deploy.Setting.IPv6 = True):
- Pod.BaseNetwork.IPv6 must be a fully expanded /56 IPv6 network prefix. By default, RFC4193 ULA fd00::/56 prefix is used as a placeholder.
- Router Version should be set to "Latest" (default)
- It is recommended that the physical layer-3 switch be configured with OSPFv3 enabled on the Lab-Routers segment
- The Ansible controller must be IPv6 enabled, and have IPv6 transit to the DNS server
- DNS server must be IPv6 enabled
- DNS server must have IPv6 forward and reverse zones
- Within each Pod, only the following components are currently configured with IPv6:
  - Nested VyOS Router (All interfaces)
  - NSX Segments
  - NSX eBGP Peering with the Router

Recommendations

The following are recommendations based on our experience with deploying Pods:

Use a physical layer-3 switch with appropriate OSPF/BGP configuration matching the OSPF/BGP settings in your config.yml file. Dynamic routing between your Pods and your physical network will make for a better experience.
Hardware configuration of the physical ESXi host(s):
- 2 CPUs with 10 cores per CPU
- 320 GB RAM
- 1 TB storage capacity (preferably SSD). Either DAS or 10 Gbit NFS/iSCSI. More space required if multiple labs are deployed.
Virtual hardware configuration of the Ansible controller VM:
- 1 vCPU (4 vCPUs recommended)
- 8 GB RAM (16GB RAM recommended)
- Storage capacity
  - 64 GB for Linux boot disk
  - 300 GB for /Software repository (Recommend this be on it's own partition/disk)
- VMware Paravirtual SCSI controller
- VMXNET3 network adapter
Deploy the pre-configured DNS server for DNS name resolution within Pods instead of using your own.

Preparations

Configure your physical network:
- Create an Lab-Routers VLAN used as transit segment between your layer-3 switch and the Pod VyOS router.
- Configure routing (OSPFv2/OSPFv3/BGP/static) on the Lab-Routers segment.
- Add the Pod VLANs to your layer-3 switch in case you are deploying the Pod to a vSphere cluster.

Install the required software on your Ansible controller:

sudo apt update
sudo apt install python3 python3-pip python-is-python3 xorriso p7zip-full git
git clone https://github.com/rutgerblom/SDDC.Lab.git ~/git/SDDC.Lab
pip3 install --upgrade -r ~/git/SDDC.Lab/pip3_requirements.txt

Update your PATH environment variable:
```
source ~/.profile
```

Install Ansible collections on your Ansible controller:

ansible-galaxy collection install --upgrade -r ~/git/SDDC.Lab/requirements.yml

Copy/rename the sample files:

cp ~/git/SDDC.Lab/config_sample.yml ~/git/SDDC.Lab/config.yml
cp ~/git/SDDC.Lab/licenses_sample.yml ~/git/SDDC.Lab/licenses.yml
cp ~/git/SDDC.Lab/software_sample.yml ~/git/SDDC.Lab/software.yml
cp ~/git/SDDC.Lab/templates_sample.yml ~/git/SDDC.Lab/templates.yml

Modify config.yml and licenses.yml files according to your needs and your environment
Create the Software Library directory structure and set folder permissions using:
```
ansible-playbook --ask-become-pass ~/git/SDDC.Lab/utils/Util_CreateSoftwareDir.yml
```
NOTE: The above command will prompt you for the 'root' password. This is needed as the playbook needs to obtain elevated permissions to create the directory tree and change file system permissions within the Software directory.
Add installation media to the corresponding directories in the Software Library (/Software)

Tested Ansible Module Versions

Although we try our best to provide detailed instructions and information about installing and using SDDC.Lab, there are some elements that we do not have control over. One such element are all the Ansible module and components. On a few occasions, we've run into issues after performing fresh installations or upgrades to those items where things didn't work. Each time, the issue turned out to be one of the Ansible modules were updated, which caused SDDC.Lab to fail.

To assist you in identifying and troubleshooting this same issue, we created a utility called Util_GetLabInfo.sh that displays information about your current environment. You can then compare the output from this utility against "tested" environments. If you find a discrepency in the version of a given module, that might point you to locating the source of your issue(s). You can find this utility in the utils folder, and we hope this information assists you with any troubleshooting that you might need to perform.

Here is a table of "tested" environments for you to compare against:

| Date | Added By | SDDC.Lab Version | LabInfo file | |------------|-------------|------------------|--------------| | 8-NOV-2024 | Luis Chanu | v-8 | LabInfo_20241108.txt |

Upgrade Considerations

Consider the following when upgrading SDDC.Lab to a newer version.

v2 to v3
- Clone the v3 branch to its own directory. For example: git clone https://github.com/rutgerblom/SDDC.Lab.git ~/git/SDDC.Lab_v3
- As additional PIP and Ansible modules are required by v3, please follow the instructions in the "Preparations" section to ensure all of the required software is installed on the Ansible controller.
- Use copies of the v3 sample files and update these with your settings. Don't copy any v2 files into the v3 directory.
- Remove the VyOS ISO file from your software library and let the router deployment script download the latest version of the rolling release.
v3 to v4
- The "SDDC Labs" folder has been renamed to "SDDC Pods" within the config_sample.yml configuration file. It's suggested that you renamed your folder and update this as part of updating your configuration files for v4.
v4 to v5
- Use of sudo is no longer required to deploy Pods.
- With the removal of using sudo in executing Ansbile playbooks, the utils/Util_CreateSoftwareDir.yml utility has been changed to leverage the --ask-become-pass option, and thus will prompt for the root password when executed.
- Additional modules have been added, and are required in order for SDDC.Lab to work. For this reason, please follow the steps outlined in the Preparations section to update your Ansile environment.
- There have been MANY changes to the config_sample.yml file. Please make sure you update your config.yml files. Once you update your ```conf

SDDC.Lab

Install / Use

README

Table of Contents