Darwinbuild Updated 2009-02-17

Kevin Van Vechten kvv@apple.com William Siegrist wsiegrist@apple.com

1. Overview 1.1 Availability, Bug Reports, Contributions, and Discussion
2. Installation 2.1 Creating the Build Directory
3. Building Darwin Projects (darwinbuild) 3.1 Install Headers 3.2 Chroot Builds
4. Build Environment Variables 4.1 Build Aliases and Alternate Targets
5. Tips and Techniques 5.1 Private Headers 5.2 Build Tools 5.3 Static Libraries 5.4 Alternate BuildRoot Storage Appendix A. darwinxref

===========

1. Overview ===========

Darwinbuild is a collection of tools that assist compilation of the many projects contained in Darwin[1], the open source base of Apple's Mac OS X operating system[2]. Apple publishes the sources of these projects in an archive format (.tar.gz). An archive is published for each project version on Apple's site[3].

These tools will provide the proper build environment as well as help to resolve any necessary dependencies prior to building.

[1] http://developer.apple.com/darwin/ [2] http://www.apple.com/macosx/ [3] http://www.opensource.apple.com/

1.1 Availability, Bug Reports, Contributions, and Discussion

Due to the constant development of darwinbuild, the only version you should be working with is trunk of the subversion repository[1]. Trunk should always be stable enough for building projects. If you find a problem or have a enhancement in mind, file a ticket at the website[2].

Discussion about darwinbuild, or building Darwin projects in general, should take place on the darwinbuild-dev[3] mailing list.

[1] http://svn.macosforge.org/repository/darwinbuild/trunk [2] http://darwinbuild.macosforge.org/report/1 [3] http://lists.macosforge.org/mailman/listinfo/darwinbuild-dev

=============== 2. Installation

You can install the latest version of darwinbuild using MacPorts.

% port install darwinbuild

Or, you can also install by checking out the source and building darwinbuild yourself. The included Xcode project will compile the tools and install them into the location specified by the PREFIX environment parameter. If the PREFIX parameter is not specified, the files will be installed into /usr/local.

By default, darwinbuild will build for x86_64 and i386. You can override this with the ARCHS environment variable. However, building trunk for ppc is not supported since trunk is targeted for Darwin 10. The release branches for Darwin 8 and 9 do support building for ppc.

To build and install darwinbuild to the boot partition:

% cd <path to top of source> % sudo xcodebuild install DSTROOT=/

2.1 Creating the Build Directory

After installation, you must initialize the build directory using darwinbuild. Assuming you wanted to build projects from 9G55 in your home directory:

mkdir ~/9G55

cd ~/9G55

sudo -s

darwinbuild -init 9G55

Creating build root disk image ... Attempting to download http://svn.macosforge.org/repository/darwinbuild/trunk/plists//9G55.plist ... Download complete Attempting to download http://svn.macosforge.org/repository/darwinbuild/trunk/plists//9F33.plist ... Download complete ... Attempting to download http://svn.macosforge.org/repository/darwinbuild/trunk/plists//9A581.plist ... Download complete Initialization Complete

ls

.build Headers Logs Roots Sources Symbols

After initialization, the build directory will contain the following: .build contains private data for the darwinbuild system Headers contains the resulting header files from previous builds Logs contains logs of previous build attempts Roots contains the finished products of previous successful builds Sources contains sources downloaded from the web Symbols contains debug symbol versions of previous build products

When using the darwinbuild command, it is necessary that the current working directory be this build directory, or alternatively, that the DARWIN_BUILDROOT environment variable be set to the absolute path of the destination directory. The example usages in this document assume the current working directory is this build directory.

========================================= 3. Building Darwin Projects (darwinbuild)

To build a Darwin project, for example xnu, the darwinbuild script can be used in the following manner:

darwinbuild xnu

The darwinxref tool is consulted to find the version that corresponds to the build specified when the build directory was initialized. It is necessary to run the darwinbuild tool as root so that projects can set the proper ownership and permissions on the build results.

darwinbuild first looks in the Sources directory for a directory containing the sources to be built (Sources/xnu-517.11.1), or a .tar.gz archive containing the sources (Sources/xnu-517.11.1.tar.gz). If neither is found, darwinbuild will attempt to download the sources from the web.

If it does not already exist, a BuildRoot directory will be created. This is where the build will actually take place. During the build, darwinbuild will change the root directory to BuildRoot (see the chroot(8) man page for details). Darwinbuild is capable of copying the required tools, libraries and headers from the Roots directory into the BuildRoot prior to building. If a necessary dependency is not found in the Roots directory, it will be downloaded from the web.

The build output will be written to the console, and additionally logged into a file in the Logs directory. The above example produces the following file: Logs/xnu/xnu-517.11.1.log~1

The ~1 indicates that this log file corresponds to the first attempt to build xnu version 517.11.1. Each subsequent attempt will add one to this build number.

If the build succeeds, the finished product will be copied out of the BuildRoot directory and into the Roots directory: Roots/xnu/xnu-517.11.1.root~1 After the copy, darwinbuild traverses the directory and records all files found in the darwinxref database. This makes it possible to query which project a file is produced by. When a Mach-O executable, library, or bundle is found during the traversal, the dynamic library load commands are recorded in the darwinxref database. This makes it possible to query which additional projects are required to run an executable produced by the project.

Additionally, any products containing debug symbols will be placed into the Symbols directory.

3.1 Install Headers

Passing the -headers flag to darwinbuild will start an alternative build style where only the project's headers are produced. This is useful for working around various circular dependencies while compiling low level projects like xnu, cctools, Libc, etc.

The results of a -headers build are placed in the Headers directory, in the same style as the Roots directory for a regular build. For example: % sudo darwinbuild -headers IOKitUser % ls Headers/IOKitUser/IOKitUser-184.hdrs~1

3.3 Logging Dependencies

Passing the -logdeps flag to darwinbuild will build the specified project while recording the paths of all files openened and all executables invoked. These paths are written into the Logs directory, and if the build is successful, will be imported into the darwinxref database.

============================== 4. Build Environment Variables

At the minimum, the build environment consists of the creation of SRCROOT, OBJROOT, SYMROOT, and DSTROOT environment variables. These variables contain an absolute path to a directory which must exist prior to invoking the Makefile. User configurable environment variables, such as the target architecture, can be set in the build plist file that is loaded via loadIndex. The darwinbuild script creates the aforementioned directories, sets the environment variables, and issues the appropriate make command.

Within the plist file, the RC_ARCHS variable indicates which target architectures should be included in the build. Currently only ppc and i386 are available. For each architecture in RC_ARCHS, and equivalent RC_{arch} should be set to YES (i.e. RC_ARCHS="ppc" RC_ppc=YES).

The RC_NONARCH_CFLAGS specify additional command line flags to be passed to the C compiler during the build. On Darwin, the -no-cpp-precomp flag should be passed since cpp-precomp is not available. During the build, the RC_ARCHS and RC_NONARCH_CFLAGS variables will be combined to create the RC_ARCHS environment variable; a -arch flag will be appended for each architecture listed in RC_ARCHS.

The MACOSX_DEPLOYMENT_TARGET variable should match the major Mac OS X release that is being targeted, such as 10.2 or 10.3.

4.1 Build Aliases and Alternate Targets

Some projects may produce different results based on the contents of the RC_ProjectName variable. When the same source archive is used to create more than one component of Darwin, it's referred to as a build alias. Because of this, it is important to always provide accurrate project name and version information in the environment. By default, darwinbuild sets these variables appropriately.

By default, "install" (for non-headers builds) is the first argument passed to the make tool (gnumake or xcodebuild). However, some projects produce different results based on alternate targets. If the project in the property list contains a "target" attribute, that string will be passed instead.

====================== 5. Tips and Techniques

5.1 Private Headers

Many open source header files are not present in a standard Mac OS X install. These "private headers" are not needed by application developers, and are not part of any SDK officially supported by Apple. However, Darwin projects are part of the operating system itself and these headers are required. The darwinbuildheaders command uses the darwinxref tool and darwinbuild -headers commands to produce all header files for a giv