Simplified script to cross-compile Python-4-Android for both ARM NEON and x86

I've created a simple script to cross-compile Python-4-Android for both the ARM NEON AND x86 architectures. This script only works with the patched Python-4-Android archive you can download from this post. I successfully ran the script on a Ubuntu 13.04 machine using the Android NDK version r9c.

First download both the make.sh script and python-for-android.tar.bz2 files in the same directory:

• python-for-android.tar.bz2
• make.sh

Cross-compiling Python-4-Android for ARM NEON

To cross-compile for ARM NEON, run make.sh with the following options:

$ make.sh -x -n /path/to/android-ndk-r9c -a armeabi-v7a-NEON -t arm-linux-androideabi-4.8

Cross-compiling Python-4-Android for x86

To cross-compile for x86, run make.sh with the following options:

$ make.sh -x -n /path/to/android-ndk-r9c -a x86 -t x86-4.6

ndk-gdb error: Could not extract package's data directory. Are you sure that your installed application is debuggable?

The Problem

You have a phone running Android 4.3 and you are trying to debug your native C++ application using the Android NDK version r9c with ndk-gdb.
When you enter the ndk-gdb command you get the error:

$ ndk-gdb  
ERROR: Could not extract package's data directory. Are you sure that
       your installed application is debuggable?

 
You can find a description of this problem in Android Issue number 58373.
I encountered this problem with a Samsung Galaxy S4 running Android 4.3. 

The Solution

The easiest solution for me was to root the device and change the permissions of the run-as command to run as root. You can achieve this by running the following commands using the ADB shell:

$ su -


# mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
# chmod 4750 /system/bin/run-as
# mount -o ro,remount -t yaffs2 /dev/block/mtdblock3 /system

Once upon a time there were Bell Labs...

Today is another step closer to the end of an era for the old AT&T Bell Labs (now called AT&T Labs). Our valued colleague David Korn (inventor of the famous Korn Shell - ksh) is leaving the labs, and he sent us the following farewell message in a form of (what else?) a Korn Shell script:

cd /tmp
if [[ $(print hello) != hello ]] 2> /dev/null
then echo 'You should use ksh'
 alias print=echo
fi
trap 'rm mvs2*' EXIT
trap 'print "$year: dgk raptured to dgk@xxxxxxxxx"' TERM
for x in 76 78 80 81 83 86 87 88 89 90 91 93 95 99 100 103 107 108 110 112 113
do case $((year=1900+x)) in
 1976) print "$year: Hello Bell Telephone Labs, Holmdel; Goodbye NYU";;
 1978) print "$year: create 'formshell' for RBCS from Bourne shell";;
 1980) print "$year: leave Bell Telephone Labs and return to NYU";;
 1981) print "$year: return to Bell Telephone Labs Murray Hill Bldg 5"
  print "$year: meet erg north dgb and kpv";;
 1983) print "$year: first version of ksh with editing and job control";
  print "$year: dgb becomes my supervisor"
  print "$year: UNIX for apollo with erg and ed fisher";;
 1986) print "$year: compete with kpv for best malloc library";;
 1987) print "$year: win best one liner in obfuscated C contest"
  print "$year: began long and continuing collaboration with gsf";;
 1988) print "$year: KornShell 88 book published";;
 1989) print "$year: 3d file system with gsf and ekrell";;
 1990) print "$year: sfio with kpv";;
 1991) print "$year: begin ast library with gsf";;
 1993) print "$year: first version of ksh93";;
 1995) print "$year: 1993 KornShell book published"
  print "$year: first version of UWIN"
  print "$year: Bell Labs split to AT&T Labs and Lucent";;
 1997) print "$year: move to Florham Park";;
 1999) print "$year: y2k testing software created";;
 2000)  print "$year: ast software becomes self documenting";;
 2003) > mvs2pc; print "$year: create mvs2pc";;
 2007) print "$year: Universal Biller invoicing runs with msv2pc";;
 2008) print "$year: UWIN merged into VizGems written in ksh by Dr. ek"
  print "$year: merged mfcobol compiler into mvs2pc";;
 2010) print "$year: distributed shell using coshell";;
 2012) mv mvs2pc mvs2cloud; print "$year: rename mvs2pc to mvs2cloud";;
 2013) kill -TERM $$;;
 esac
done
exit

How to debug Android Native Code with Eclipse

This blog summarizes the steps needed to set-up your Eclipse environment to support the debugging of an Android native application written in C/C++. It's taken from an old Carlos Suoto's web page that doesn't seem to exist anymore.

1. Pre-Requisites

• Make sure you compile your C/C++ with the "-g" option (or use -DCMAKE_BUILD_TYPE:STRING="Debug" if you use cmake).
• Make sure APP_OPTIM is set to "APP_OPTIM:=debug" in Android.mk and Application.mk.
• Make sure build/core/build-binary.mk in Android NDK doesn't strip executables. For example patch android-ndk-r8e/build/core/build-binary.mk to the following:
  
  --- build/core/build-binary.mk.orig     2013-08-21 11:06:39.818329442 -0400
  +++ build/core/build-binary.mk  2013-08-21 11:13:11.877214361 -0400
  @@ -485,10 +485,16 @@ $(LOCAL_INSTALLED): PRIVATE_DST       :=
   $(LOCAL_INSTALLED): PRIVATE_STRIP     := $(TARGET_STRIP)
   $(LOCAL_INSTALLED): PRIVATE_STRIP_CMD := $(call cmd-strip, $(PRIVATE_DST))
   
  +ifeq ($(APP_OPTIM),debug)
  +$(LOCAL_INSTALLED): $(LOCAL_BUILT_MODULE) clean-installed-binaries
  +       @$(HOST_ECHO) "Install        : $(PRIVATE_NAME) => $(call pretty-dir,$(PRIVATE_DST))"
  +       $(hide) $(call host-install,$(PRIVATE_SRC),$(PRIVATE_DST))
  +else
   $(LOCAL_INSTALLED): $(LOCAL_BUILT_MODULE) clean-installed-binaries
          @$(HOST_ECHO) "Install        : $(PRIVATE_NAME) => $(call pretty-dir,$(PRIVATE_DST))"
          $(hide) $(call host-install,$(PRIVATE_SRC),$(PRIVATE_DST))
          $(hide) $(PRIVATE_STRIP_CMD)
  +endif

1.1. Install the Eclipse Sequoyah plugin

1. In Eclipse go to Help -> Install New Software
2. Click on the Add button
3. Enter 'Sequoyah Metadata Repository' in the Name field
4. Enter 'http://download.eclipse.org/sequoyah/updates/2.0/' in the Location field
5. If you still see the "There are no categorized items" message, uncheck the "Group items by category" radio button
6. Select "Sequoyah Android Native Code Support" and install the plugin

1.2. Convert The Android Java Application into C/C++ Project

You must convert your Java project to C/C++ using the Sequoyah plugin or else you won't be able to see the configuration options to set the proper debugger settings.

1. In Eclipse select the Android Java project you need to convert
2. Right click with the mouse and select Android Tools -> Add Native Support

1.3. Other Pre-Requisites

1. The platform must be Android 2.2 (android-8) or later
2. The ndk version must be r4b (it contains bugfixes to ndk-gdb that are necessary) or later
3. Eclipse CDT 7.0 or newer must be installed
4. The AndroidManifest.xml must have the property of the application node android:debuggable="true"
5. The build must have been done with the ndk-build (if using the Sequoyah Android components, it will be automatic)

2. Configurations

• 01) Create a debug configuration for an Android application (can be done with Eclipse or MOTODEV Studio)
• 02) Create a debug configuration for a C/C++ application

03) Set the following properties:

• 04) The process launcher must be the Standard Process Launcher. This is selected at the bottom of the Main tab:

• 05) On the "Main" tab:
  the Field C/C++ Application: $PROJECT_PATH/obj/local/armeabi/app_process
  
  

  

  
  
• 06) On the "Debugger" tab:
  
  • field Debugger: gdbserver
  • 
    On the "Main" subtab:
    
    
  • 07) GDB debugger: $NDK_PATH/build/prebuilt/$ARCH/arm-eabi-$GCC_VERSION/bin/arm-eabi-gdb
  • 08) GDB command file: $PROJECT_PATH/obj/local/armeabi/gdb2.setup
    [Windows users] Uncheck the "Use full file path to set breakpoints" option
  • 
    
    On the "Connection" subtab:
    
    
    

    

    
    
  • 09) Type: TCP
  • 10) Hostname or IP address: localhost
  • 11) Port number: 5039 
  
  

  3. Instructions

• Open the ndk-gdb script that came with the android NDK and comment the last line (we are not calling the usual gdb client, but we will attach an Eclipse gdb session instead):
  
  

  •     # $GDBCLIENT -x $GDBSETUP -e $APP_PROCESS
    

• Insert a breakpoint in your Java code, preferably after all System.loadLibrary() calls. (To make sure that the debugger is correctly attached to the Java process)
• Launch the android debug and wait for it to reach the breakpoint
• From a Terminal session, in the project folder, run the modified ndk-gdb command. It should not attach to an gdb client, but call the gdbserver on the emulator and open a TCP port for connection (or in alternative if you have an Android device connected to your USB port, the ndk-gdb script will run gdbserver on the device itself).
• In the $PROJECT_PATH/obj/local/armeabi/, modify the gdb.setup file, removing the target remote:5039 statement. (For some reason, the Eclipse GDB session does not like this statement being done in the commands file). Rename this new file to gdb2.setup. This step need to be run just once, on the first debug session.
• Launch the C/C++ Application debug and wait for the Eclipse GDB session to fully connect to the emulator's gdbserver instance.

After following these steps, one can continue to debug the application as usual, using the "continue" option to let the execution flow until the next breakpoint is hit or by using the usual "step-in" to execute each statement individually. Setting a breakpoint on a Java statement that calls a native function through JNI and stepping into will place the user at the beginning of the native code.
Another way to set breakpoints in the C/C++ code is from Eclipse to click on File -> Open File and browse to the location of your source code. Then double-click on the line where you want to set the breakpoint.

How to compile libogg for Android

To compile libogg for Android, you just need to create an appropriate jni/Android.mk makefile with the minimum set of files and options needed to compile libogg for Android.

1. Download ogg from http://downloads.xiph.org/releases/ogg/ (for example I downloaded libogg-1.1.4.tar.gz)
2. untar the file in your local directory:
   
   $ tar zxf libogg-1.1.4.tar.gz
   
3. create a jni directory and put the following text into a file named jni/Android.mk:
   
   LOCAL_PATH := $(call my-dir)
   
   include $(CLEAR_VARS)
   LOCAL_MODULE    := ogg
   LOCAL_CFLAGS    := \
       -DFIXED_POINT -DUSE_KISS_FFT -DEXPORT="" -UHAVE_CONFIG_H \
       -D_ANDROID
   LOCAL_C_INCLUDES := \
       ../libogg-1.1.4/include
   LOCAL_SRC_FILES := \
       ../libogg-1.1.4/src/bitwise.c \
       ../libogg-1.1.4/src/framing.c
   include $(BUILD_SHARED_LIBRARY)
   
4.  Now your directory should look like the following:
   
   $ ls
   jni/  libogg-1.1.4/  libogg-1.1.4.tar.gz
   
5. change directory to jni and run ndk-build:
   
   $ cd jni; ndk-build V=1

That should do it.

How to compile busybox with Android NDK for both ARM and x86 architectures

I was looking for a way to run busybox on a Motorola RAZRi with an x86 Intel Atom processor but I couldn't find any Android app from the Google play market that was running on the phone (when running any busybox command nothing happened and I was immediately getting the prompt back).
So I decided to cross-compile busybox for Android myself. Now you will find other posts claiming to have done so by downloading an ARM-cross compiler from codesourcery.com but I had two problems with that approach:

1. I needed an x86 cross-compiler and,
2. I read various complaints of people not being able to run various busybox commands with this method.

Cross-compiling busybox with the Android NDK

My only option was to use the Android NDK itself, since it comes with full support of the ARM, x86, and MIPS architectures. I ended up manually configuring busybox by eliminating any module that would not cross-compile due to missing header files and/or libraries in the Android NDK.
To cut the story short, I managed to cross-compile busybox using the Android NDK compilers for both ARM and x86 architectures (I didn't try MIPS since I don't have a device with a MIPS processor to test the resulting binaries).
To build busybox yourself, download the busybox-android.patch, busybox-android.config, and build.sh script and run the latter with the following options:

$ build.sh <android NDK dir> <NDK toolchain> <Android ABI> <Toolchain compiler version> <Android Native API Level> <Install prefix>

The script will download the busybox version 1.21.0 source code, apply the busybox-android.patch and configure busybox for the right architecture.
For example if you have your Android NDK installed under /home/android/android-ndk-r8d, to build busybox for the Intel x86 architecture call the build.sh script like the following:

$ build.sh /home/android/android-ndk-r8d x86-4.7 x86 4.7 android-14 /home/android/busybox-x86

and the cross-compiled busybox binaries will be installed under  the  /home/android/busybox-x86 directory. Similarly if you want to build busybox for the ARM processor, run the following:

$ build.sh /home/android/android-ndk-r8d arm-linux-androideabi-4.7 armeabi-v7a  4.7 android-14 /home/android/busybox-arm

Download busybox binaries

The busybox binaries for x86 (compiled using the above procedure), can be found here.

Installing busybox

The above procedure was tested with release r8d of the Android NDK and for both ARM and x86 devices.

Pre-requisites

1. Your phone must be rooted (for example check out my blog How to root Motorola RAZRi XT890 running Android 4.0.4)
2. You must activate USB debugging mode on your device by going to Settings -> Developer Options -> USB Debugging and selecting the corresponding check box.

Installation Procedure

To install busybox on an x86 Android (rooted) device, I run the following commands with the phone connected to the computer via USB:

# logon to the phone and become root to mount the /system file system read/write
$ adb shell
shell@android:/ $ su -
shell@android:/ # mount -o rw,remount -t ext4 /dev/block/system /system
shell@android:/ # chmod 0777 /system/xbin
shell@android:/ # exit
shell@android:/ $ exit
$

then change directory to the location of the busybox binary file and push it to the phone:

$ adb push busybox /system/xbin/

log back on as root and change permissions appropriately:

$ adb shell
shell@android:/ $ su -
shell@android:/ # chmod 755 /system/xbin/busybox
shell@android:/ # chmod 0755 /system/xbin
shell@android:/ # exit
shell@android:/ $ exit
$

Installing busybox on an ARM Android device is very similar, only the mounting instruction line changes:

$ adb shell
shell@android:/ $ su -
shell@android:/ # mount -o rw,remount -t yaffs2 /dev/block/mtdblock3 /system
shell@android:/ # chmod 0777 /system/xbin
shell@android:/ # exit
shell@android:/ $ exit
$
$ adb push busybox /system/xbin/
$ adb shell
shell@android:/ $ su -
shell@android:/ # chmod 755 /system/xbin/busybox
shell@android:/ # chmod 0755 /system/xbin
shell@android:/ # exit
shell@android:/ $ exit
$

How to build the gcc Fortran cross-compiler for Android (ARM and x86)

If you need to cross-compile for Android a program written in Fortran, you know already that the official Android NDK does not come with the gfortran compiler, and if like me you need to port to Android code that depends on Fortran (such as the lapack libraries), you are out of luck.
Fortunately I managed to compile the gcc Fortran cross-compiler with the help of  Mike Long's blog. As an added bonus, I managed to build gfortran 4.8.0 not only for the ARM but also for the x86 toolchain, as well as update the script to the latest Android NDK (currently r8d).
What you need to do is the following:

1. download the fortran4android shell script.
2. download the ndk-r8d fortran patch.
3. run the script in your directory:
   
   $ fortran4android
4. wait, wait, wait, ... and you will see in the android-ndk-r8d/toolchains directory two new toolchains called arm-linux-androideabi-4.8.0 and x86-4.8.0
5. Enjoy your new gfortran compiler.

Note that I've successfully tested the script only on Ubuntu 12.04.

Addendum of August 30, 2013

I now have available a patch for the newer android-ndk-r9. To use this instead of the patch above:

1. download the ndk-r9-fortran-patch
2. change the fortran4android script by replacing "r8d" with "r9" everywhere
3. run the fortran4android script in your directory.

 Note that r9 of the NDK already comes with a 4.8 gcc toolchain and to keep it separated, the fortran4android script will still generate in output a "4.8.0" toolchain.