On the previous post, we got the AOSP code into our environment and learned how that process works. The repo tool and the Manifest project goes hand-in-hand, managing the 200+ git repositories, all part of the AOSP.
Now that we have all the code locally, we need to get all those lines of code into a nice package we can eventually install on a device. The nice guys in Google brought us some nice utilities to help with that. To initialize your environment with those new functions and environment variables, you need to run the command:
$ source <working dir>/build/envsetup.sh
Note: I’m using Mac OS with “oh-my-zsh” and “zsh” as my default shell. When running this command from a shell other than “bash”, you’ll get a warning that only bash is supported. From my experience, I can tell that I never ran into a problem. If you know otherwise – let me know, otherwise – I highly recommend this configuration regardless of AOSP work.
Now, we can get this party started..
There are several devices we can build our new Android ROM for (remember what ROM is?), with different feature combinations and build types that we can use. In order to pick the desired configuration, we’ll use lunch. The lunch command can take the configuration name as an argument, or picked by the user from a list. A configuration name is structured this way:
PRODUCT – This part describes the set of modules to be included, among other configurations. We can include only some of the packages in the system, or all of them, depending on the “flavor” we want our new ROM to be. A “package”, for this matter, can be an app, a system component, sounds, locales, fonts or even just a regular files that are copied to the final product. We can also set different system properties for a specific product. There are several predefined products we can use:
generic – default set of packages.
full – a full set of packages, with most necessary apps and all locales
aosp – Basically inherits everything from full.
sdk – packages needed to build the SDK. I’ll elaborate about the SDK on a separate post.
There are also sub-products for specific uses. For instance, you can have full_flo to build the full product for the Nexus 7 (codename “flo”), providing you have the necessary drivers (more on that later). To see what’s included in each product, check the .mk files under <working dir>/build/target/product and also under /device/<vendor>/<device codename>.
BUILDTYPE – Selects the modules to install and set some default settings or behaviors of the product. Each variant can accept only modules who set, in their Android.mk file, the LOCAL_MODULE_TAGS to at least one of the tags it approves (tag examples: debug, samples, tests and more) . The valid build types are:
eng – this variant will have some development tools and predefined settings that developers need (e.g. USB debugging enabled).
user – this is the variant for a product that suppose to go out to the users. Will use only modules that the end-user will need and more restricted settings.
userdebug – A combination of the user and eng variants. It’s intended for users testing your environment, but not developing it.
You can see in more detail the different handling for these variants in the <working dir>/build/core/main.mk file, by following the use of the TARGET_BUILD_VARIANT env variable.
After selecting our preferred product and build variant, we can apply it using the lunch command. For this demonstration, I want to run my new ROM on an emulator and have access to every advanced feature available. For that, I’ll use the “full-eng” configuration and I’ll apply it by running:
$ lunch full-eng
Build flash-able images
The basic build artifacts are system images, one for each partition on the device. To build our images, we need to use the make command. To build my ROM on my computer, I’ll use this command:
$ make -j16
The -jN switch is defining the amount of threads to use when making the project. That’s a standard “GNU make” switch, not unique to AOSP builds. To get the maximum out of your machine, the recommended value for N is CPU_cores*2, so in a single quad-core CPU machine, you should probably use N=8, but you can try and play with the values to get the most out of your machine. My Macbook Pro has one quad-core CPU with Hyper-Threading, meaning it can run 2 threads simultaneously on each core, providing 8 “virtual” cores. I use mostly N=16 and sometimes N=8, depends if I’m still working on the laptop while building.
The build process is long, could take anywhere between 30mins to 3 hrs. We’ll talk more about time comparisons and performance boosting later on. If you need something to do while it’s building, I can recommend watching my new favorite TV show “Rick and Morty”, it’s a combination of “Familty Guy” and “Back to the Future” but also cleaver as “South Park”. You can stream it for free from their official website and it works smoothly even if using make -j16 (tested it myself!).
After the process is complete, the emulator’s path is automatically added to your terminal’s PATH, meaning we can start it by running:
To check that I’m looking at the image that I just built, I’ll go to the “About Phone” screen and check the “Build Number”.
I can see the build configuration that I selected – “full-eng”, and my name as the user who built it. I can also see the date/time of the build and that I used the default test key to sign all the modules (which, of course, needs to be changes before releasing). Check the <working dir>/build/core/version_defaults.mk file to see how the build number is formatted.
Building for actual devices
make clobber# Both are basically the same
Go with the Flo
Bootloader, Fastboot and Download Mode
The “Build number” has our lunch target (aosp_flo-eng), my name, the signing keys and the date/time of the build. This screen is the best place to check if your flash process was successful.
Creating an OTA Package
In contrary to image files, where the entire partition is replaced with the content of the image files, an OTA package performs an update to the system’s files. It’s smaller in size and can be incremental from the last build. You can check the <working dir>/build/tools/releasetools/ota_from_target_files file, to see how the OTA package is actually created
To create an OTA package, we need to first pick a build configuration, using lunch as before, and then use
$ make otapackage -j16
After the build finishes, the env variable $OUT or $ANDROID_PRODUCT_OUT will be initialized to the path where the product can be found, that’s where the OTA zip file will be. It should be around 200 MB in our case, less than the sum of all image files (> 400MB). Installing the OTA can be done using the Recovery system.
What is a Recovery you ask?
Remember the bootloader I mentioned earlier? Well..it can boot our device into the Android OS, that we all know and love, or into another partition with a Recovery console. The latter has tools to help the user recover or repair a problem he’s having with the OS. The basic recovery on AOSP is very limited, it offers basic operation such as deleting user data, upgrading the firmware and is being controlled using the volume up/down and power keys. You can use a custom recovery such as the popular ClockworkMod Recovery, which supports more functions and touch screen controls. The easiest way to flash a new custom recovery, is using the app ROM Manager, but there are other methods simple as a one fastboot command. Different devices require different steps to do that, search the appropriate method for your device. You can read more about the Recovery here.
In both Stock and Custom recoveries we can flash a new OTA using the adb sideload method. The stock recovery will check the package’s signature, a check you can bypass on a custom recovery. So, if you didn’t sign your OTA package correctly – you can use a custom recovery instead.
If the previous ROM on the device was different than the one you installing, it’s encouraged to do a factory reset and clear the cache, to clean up the place and avoid upgrading issues. This can be done from the Recovery console as well. After getting the device into “sideload mode”, we install the OTA package with this command:
$ adb sideload <path to OTA package>
The sideload command copies the OTA file to the device and install it. After the process completes – our new ROM is now installed. It is also possible to write code that does the updating for us, you can read about the RecoverySystem class and take an example from Cyanogenmod’s updater
An important note for Mac users: After building an OTA package for my Nexus 7, I noticed that the WiFi isn’t working at all. After doing a bit of research, it seems like there’s a problem with the WiFi driver on OTA packages if built on Mac. Creating the OTA package on a Linux system resulted in a fully working WiFi. I found another person with the same problem as I’m having, but for a Galaxy Nexus device, which means this problem may occur on other devices too.
Build time and performance boost using CCache
-s – See statistics of the cache. The amount of files, total size and more.
-C – Clears the cache
I highly recommend using the ccache for your builds. It really is improving the build time and I never had any problems with using the cache. But then again – I don’t normally change C/C++ files in the AOSP. If you find yourself in a weird compilation problem, and you want to just clean everything up, use make clean and also run this to clear the ccache:
$ <working dir>/prebuilts/misc/darwin-x86/ccache/ccache -C