Android Cookbook: AOSP ROM Building 102

Disclaimer [2013 Dec 09]

The following sections were developed approximately a year ago on Jellybean. In that time we have had multiple versions of AOSP and the processes have likely changed. The processes (recipes) listed below are relatively close, but need to be validated / updated. That is expected to happen before the end of the calendar year. So the message is that these recipes are not complete accurate and if you use them, expect some frustration – tomwwolf

Overview

Building from the Android Open Source Project (AOSP) Codebase is relatively well documented at the AOSP site, but from time to time the AOSP site is lacking in something. In many case they provide some version specific process without guidance on how to adapt in a general manner. In some cases they simply skip details –  perhaps details they consider trivial.  The procedures below do not provide the progressive walk through you will find at the AOSP site, but it does provide some self contained recipes that may be more concise than the AOSP site and/or provide information that may help you figure it out yourself. Note – Consider the AOSP site to be the authoritative site for all things about building AOSP – so if you discover a conflict between these processes and the AOSP site, go with the AOSP site.

The sections below provide a number of standalone recipes that are part of the AOSP build process. As a general rule these recipes do not provide prerequisites, and it is assumed the user knows when and where these recipes should be applied. If you are not sure, be appropriately cautious.

Build Platform Oracle Java Install

Overview

This process defines the installation process for installing the Sun / Oracle Java 6 JRE / JDK on the build platform. Although this would seem to be intuitive and it should be relatively easy, the current licensing / litigation environment around Oracle Java has turned this into a very indirect process and occasionally clumsy process.

I am documenting this process here rather than referencing the appropriate JDK Install AOSP page, since the instructions on that page seem to be written to a specific version of Ubuntu, and that may be problematic. Their procedures may work for newer versions of Ubuntu, but it may not. The instructions below should work for any version of Ubuntu. Note that it may be tempting to install Java 7 or even Java 8 – but the AOSP team has indicated that their reference install is based on Java 6, and that it does matter. They have also indicated that Oracle Java is necessary for everything to work correctly. So to be safe stick with Oracle Java 6.

The current version of the Java 6 JDK is ‘6u45’ and is reflected in the instructions below. The current version you find at the Oracle site will likely have a different postfix number. I recommend using whatever is the newest version of Java 6. If you use an different version, modify accordingly. At this time it is located at http://www.oracle.com/technetwork/java/javasebusiness/downloads/java-archive-downloads-javase6-419409.html and titled ‘ jdk-6u45-linux-x64.bin’.

Process

  1. Remove the OpenJDK packages.
cd ~/
sudo apt-get remove openjdk*
sudo apt-get autoremove
  1. Download the newest Java 6 JDK from java.sun.com, with the ‘.bin’ extension. By default it should be in the Downloads directory.
cd ~/Downloads
chmod +x jdk-6u34-linux-x64.bin
./jdk-6u34-linux-x64.bin
  1. This will expand the JDK into a directory of the same name in the Downloads directory, and then attempt to open a license agreement in a browser. You can cancel out of this at this point.  We will now move the JDK to a more useful location.
mv jdk1.6.0_34 ../bin/.
  1. Note we are still leaving the JDK inside of the user directory. The next thing we need to do is edit the ‘.bashrc’ file to include the JDK in the path.
cd ~/
vim .bashrc
  1. Scroll to the bottom, and append the following line to the end of the file (inserting the appropriate username and JDK directory name).
PATH=/home/<username>/bin/jdk1.6.0_34/bin:$PATH
  1. Open a new terminal window and enter
java –version
  1. It should reflect some variation of Oracle Java 1.6. If not review the steps and correct.

Validation

cd ~/
java -version

References

Build  Platform Base Package Installation

Overview

This process defines the installation of the core platform build packages and the AOSP repo script. The AOSP site defines their reference install as Ubuntu Lucid (10.04 LTS), but I am going to use something significantly newer – Ubuntu 12.04 LTS, also known as Precise.

Process

  1. Install Oracle JDK 6 (see above).
  2. Install baseline packages. Open a terminal window and enter:
sudo apt-get install build-essential
sudo apt-get install python
sudo apt-get install git
sudo apt-get instal gnupg flex bison gperf build-essential \
 zip curl libc6-dev libncurses5-dev:i386 x11proto-core-dev \
 libx11-dev:i386 libreadline6-dev:i386 libgl1-mesa-glx:i386 \
 libgl1-mesa-dev g++-multilib mingw32 tofrodos \
 python-markdown libxml2-utils xsltproc zlib1g-dev:i386
sudo ln -s /usr/lib/i386-linux-gnu/mesa/libGL.so.1 /usr/lib/i386-linux-gnu/libGL.so
  1. Get Repo – Although the Google build instructions place the repo script in the ~/bin directory, we also copy it to /usr/local/bin in order to more effectively support a mirror repository.  Note – As part of the build process, repo will occasionally update. Since the mirror repository is implemented as su and the local repository as your non-su user,  we have two separate copies of repo in the respective locations – instead of a soft link to the primary.
mkdir ~/bin
curl http://commondatastorage.googleapis.com/git-repo-downloads/repo > ~/bin/repo
cd ~/bin
chmod a+x ./repo
sudo cp ./repo /usr/local/bin/.
  1. Edit  “.bashrc” in your home directory and append the PATH statement to make the ~/bin directory path change persistent(inserting the appropriate username). It is fine if there are multiple PATH statements in the .bashrc file, as long as it includes ‘:$PATH’ on the right side of the expression to carry forward the existing path.
PATH=/home/<username>/bin:$PATH

References

Build Platform USB Configuration

Overview

This process defines the configuration changes needed so that your build system recognizes your Nexus device, and allows your build user access to that device.

Process

  1. Go to http://source.android.com/source/initializing.html and copy the USB content identified for the “/etc/udev/rules.d/51-android.rules”. I am referencing you to that site, since that will be the most current content for Nexus USB devices. Note – the USB device content (on 2013Dec06) is missing for Nexu4, Nexus5, and the 2013 Nexus 7 devices.
  2. Note– If you are using a device not listed in this content, we are going to need to get the pair of USB IDs for the device before we can generate this file. Open a terminal, and enter ‘sudo dmesg’ and note the final entries. With the Android device powered up, plug it into the system, and run ‘sudo dmesg’ again. Note the new content, and look for some content that looks like “idVendor=18d1”. That number (18d1) is the USB id for ADB, since the device OS is powered up. Switch the device to the bootloader, and repeat the process to get the corresponding USB ID for fastboot. Create an entry in the file below where you copy the format of the other devices, but substitute the USB IDs for your device.
  3. Create the AOSP device file named “/etc/udev/rules.d/51-android.rules”.
cd /etc/udev/rules.d
sudo vim 51-android.rules
  1. From the AOSP site, and copy that content into this file. Each device has two lines – One for ADB mode and one for Fastboot mode. Technically it is only necessary to install the lines that appropriate to your device, but generally including all Nexus devices is better practice. Note – as new Nexus devices are announced, this list *should* be updated at the source.google.com reference.
  2. Update the content and replace <username> with the name of your build user, leaving the quotes. For example, if your build user name was ‘joeuser’, the result is OWNER= “joeuser” at the end of each line. After completion, write the file out and exit.
  3. Restart udev and adb services. In a terminal windows:
sudo service udev restart
sudo killall adb
  1. If the Nexus device is not booted to Android, do so now. When it is done, plug the Nexus device into the build platform USB. The device should be listed after this command. Note – the screen must be unlocked, debug mode enabled and there will be a dialog asking for ADB permission.
adb devices
  1.  If the device is not recognized, edit the ‘/etc/udev/rules.d/51-android.rules’ file. Change the appropriate device adb MODE to 0666, and restart udev and adb. If there are still issues check the ownership of the same file and confirm that it is ‘root:root’.

Validation

adb devices

References

Build Platform Local Mirror Repository

Overview

This process defines the process for creating a local mirror repository for the AOSP. This is essentially making a copy of the AOSP codebase on your system that is not used for building. In general this is good practice if you have the storage space. The mirror (at this time) takes about 14GB of storage, but it provides a few key advantages. These include: a) one time updates from the Google AOSP servers for multiple concurrent projects, b) a privilege protected local repository, c) a much faster source to build a new project from.

Note: If this is a “one and done” evaluation, do not create a local mirror – build directly from the Google AOSP repository. A mirror only makes sense if you intend on using it multiple (many) times.

Process

  1. Setup local mirror. Note that all operations are ‘su’ operations. This prevents the build user from inadvertently corrupting the mirror repository.
sudo mkdir -p /usr/local/aosp/mirror
cd /usr/local/aosp/mirror
sudo repo init -u https://android.googlesource.com/mirror/manifest --mirror
sudo repo sync
  1. Once the mirror is synced, new clients can be created from it. Note that it’s important to specify an absolute path. Note – there will be errors on the repo sync since some of the branches are vendor locked, but these can be safely ignored.
  2. Note – Update workflow requires that you update the mirror first with ‘sudo repo sync’ in the mirror directory, followed by a ‘repo sync’ in each project directory.

Reference

Initialize Build from Local Mirror

Overview

This process defines the process for initializing a build project from a local mirror at /usr/local/aosp/mirror. This is the recommended process if: a) you have sufficient hard drive space, and b) you are going to be experimenting with multiple concurrent AOSP builds. If this is a “one and done” evaluation, do not create a local mirror – build directly from the Google AOSP repository.

Note: In case it was not obvious, this procedure requires that the local mirror has already been installed and configured (covered in another recipe on this page).

Process

  1. Once the mirror repository is synced, new build projects can be created from it. Note that it’s important to specify an absolute path:
  2. Note – branches are listed under /usr/local/aosp/mirror/platform/manifest.git/refs/heads
mkdir -p ~/aosp-working
cd ~/aosp-working
repo init -u /usr/local/aosp/mirror/platform/manifest.git
repo sync –j16

Reference

Initialize Build from GoogleSource

Overview

This process defines the process for initializing a build project from GoogleSource. This process is relative slow since the repository is large, has many files, and it runs over your internet connection. I strongly recommend that if you are serious about building Android, you setup a local mirror to save yourself time and frustration.

Process

  1. Initializing the build directory from GoogleSource. This will be slow since the repository is large. Get it started and give it some time.
mkdir aosp-working
cd aosp-working
repo init -u https://android.googlesource.com/platform/manifest
repo sync

Reference

Build AOSP for ARM Emulator

Overview

This process defines how to build the baseline AOSP code for the emulator.  It is a good validation that everything is configured and working correctly.

Process

  1. Environment Setup – from inside the source tree working directory.
. build/envsetup.sh
  1. Pick a Target – my favorite test target is  ‘aosp_arm-eng’ for emulator.
lunch
  1. Building Code. The ‘j’ parameter specifies the number of make threads. There are some cases where –j1 (single threading) will have fewer faults. This can occur when there are some dependencies faults in the build process. I have not actually seen this happen in the AOSP baseline code. A rule of thumb is to specify ‘jxx’ at twice the number of cores or threads your build system supports. For example my build system has an Intel i7 quad core with 16GB of RAM, and I use –j16 for most of my builds. The default (if not specified) is –j4.
make -j16
  1. Running the Emulator: After a build, the paths and parameters are all defined so that the emulator can be launched with the completed build with:
emulator –show-kernel
  1. If the emulator or images are not found, try the following (from the build directory):
. build/envsetup.sh
lunch full-eng
emulator –show-kernel
  1. Note: On the first run of the emulator on a new build, it will take much longer since the emulator is building the system cache. Be patient and let it complete, unless the kernel messages show a kernel panic, in which case the build crashed out (indicating that there are other issues).

Reference

Using AOSP Emulator Image Portably

Overview

This process defines how to archive and move a set of build images to some other system (portably) and run those images on an emulator on that system. This enables you to have an AOSP build system – which is generally going to be a fairly high powered system, and also have a separate Android SDK system for application / tool development while using the same emulator images.

It also means that you can package and run this ROM on laptop suitable for demonstrations, or an application developer target copied to multiple a developer systems.

Process

  1. Build an emulator on the AOSP build system. Reference the recipe above.
  2. From the build directory navigate to the build images at ‘out/…’.
mkdir emulator
cd emulator
cp ../out/target/product/generic/*.img .
cp ../prebuilts/qemu-kernel/arm/kernel* .
zip <emulator-name>.zip *
  1. Copy the archive image to your Windows SDK system
  2. Expand the archive image zip file. Create a file named ‘run-emulator.cmd’ in the archive directory with the following contents:
echo on
rem – skindir must point to your skindir
set skindir=S:\Android\android-sdk\platforms\android-16\skins
set skin=HVGA
call emulator -sysdir %CD% -system system.img -ramdisk ramdisk.img -data userdata.img -kernel kernel-qemu-armv7 -skindir %skindir% -skin %skin%
pause
  1. Run ‘run-emulator.cmd’ and it should launch the emulator.

Reference

Building AOSP Kernel

Overview

This process defines the process to building a kernel for a device from source. In Android the kernel source is built from many million of lines of code. This will take a little while.

Process

  1. From inside the AOSP Build directory, make a target build directory
mkdir kernel-working
cd kernel-working
  1. Get the prebuilt cross compiler
git clone /usr/local/aosp/mirror/platform/prebuilt
export PATH=$(pwd)/prebuilt/linux-x86/toolchain/arm-eabi-4.4.3/bin:$PATH
  1. Export some build parameters.
export ARCH=arm
export SUBARCH=arm
export CROSS_COMPILE=arm-eabi-
  1. A local kernel build (e.g. Nexus 7 – grouper) can be initialized with:
git clone /usr/local/aosp/mirror/kernel/tegra
cd tegra
git checkout android-tegra3-grouper-3.1-jb-mr1
  1. Build:
make tegra3_android_defconfig
make
  1. The resulting kernel will be located at ‘arch/arm/boot/zImage’. It will need to be renamed and copied into your AOSP build tree. Note – since this is not typically a modular kernel, it should indicate ‘0 modules’ after the kernel message.
cd arch/arm/boot
mv zImage kernel-grouper-custom-2012-qaz
cp kernel-grouper-custom-2012-qaz <aosp-build-dir>/devices/asus/grouper/.
  1. This will name the kernel in a sufficiently unique way and copy it to the device build directory.
  2. In order to integrate this into the build (from the aosp-build directory):
cd <aosp-build-dir>/devices/asus/grouper
mv kernel kernel-aosp-grouper-baseline
cp kernel-grouper-customer-2012-qaz kernel
  1. Rebuild the AOSP.
  2. After the build is completed, the new kernel should be in the image files. This can be verified by the kernel version name in the ‘Settings/About Tablet’ and build date/time.

Reference

Installing Binary Drivers in Build Process

Overview

This process defines how to include Nexus binary drivers in a build image.

Process

  1. Download the binaries for your Nexus device at https://developers.google.com/android/nexus/drivers
  2. Expand the .tgz files. They should produce .sh files. Copy these .sh files to the root directory of the  build project.
repo start bin-drivers --all
repo checkout bin-drivers
  1. Run each one of the .sh files, and accept the license.
  2. Run ‘make clobber’ to ensure the binary directories are integrated in the build process.
make clobber
. build/envsetup.sh
lunch full_grouper-userdebug
make –j16
  1. Note – Installing the binary drivers through this mechanism is the official path for driver installation. Unfortunately, most Android devices do not have binary drivers available and it is necessary to extract the files from the device using ‘extract-files.sh’ in combination with ‘proprietary-blobs.txt’. This is the method used for the device trees in the CyanogenMod project.

Reference

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.