Friday, June 25, 2010

Device Information Extras

madb-xunitMy goal for Managed Android Debug Bridge is not just to port the functionality from DDMS/ADB, but to make it useful to developers.

The original Device in ddms has a dictionary for mount points, but it only has the 3 hard coded m ount points. So if a rooted device, with busybox is connected, then it will pull all the mount points on the system.

 

 

 

This gives some information like this:

rootfs			/			rootfs	ro
tmpfs /dev tmpfs rw
devpts /dev/pts devpts rw
proc /proc proc rw
sysfs /sys sysfs rw
tmpfs /sqlite_stmt_journals tmpfs rw
none /dev/cpuctl cgroup rw
/dev/block/mtdblock3 /system yaffs2 ro
/dev/block/mtdblock5 /data yaffs2 rw
/dev/block/mtdblock4 /cache yaffs2 rw
/dev/block/mmcblk0p2 /sd-ext ext2 rw
/dev/block/vold/179:1 /sdcard vfat rw

Notice how the /system mount is read-only. That is because I also have a method to set the mount point ReadWrite/ReadOnly. Again, this will really only be useful for rooted devices, but it also gives the ability to check what the current “access” of the mount is as well. So Droid Explorer can check if the mount is read-only before doing a file operation, like a delete or create, and it can change the mount to read-write if needed (and if it is "enabled" by the user).


MADB also can transfer files better then regular adb. The ddms library does not handle symlinks, but MADB does. So if you try to copy a symlink, it will resolve it and copy over the original, but still name it what the symlink is.


Pulling multiple files from the device can also easily allow developers to show the total progress for the transfer. For the unit tests, I created a simple ISyncProgressMonitor that writes how many bytes were transferred, out of the total bytes, plus how many bytes are remaining. Here is a sample output, with some data trimmed for space, of a pull of /system/app


Starting Transfer
Syncing /system/app/BugReport.apk
Transfered 23669 of 13334850 - 13311181 remaining
Syncing /system/app/EnhancedGoogleSearchProvider.apk
Transfered 89205 of 13334850 - 13245645 remaining
Transfered 154741 of 13334850 - 13180109 remaining
Transfered 220277 of 13334850 - 13114573 remaining
Transfered 274775 of 13334850 - 13060075 remaining
...
Syncing /system/app/Street.apk
Transfered 10224322 of 13334850 - 3110528 remaining
Transfered 10264332 of 13334850 - 3070518 remaining
Syncing /system/app/Talk.apk
Transfered 10329868 of 13334850 - 3004982 remaining
Transfered 10395404 of 13334850 - 2939446 remaining
Transfered 10460940 of 13334850 - 2873910 remaining
Transfered 10526476 of 13334850 - 2808374 remaining
Transfered 10592012 of 13334850 - 2742838 remaining
Transfered 10657548 of 13334850 - 2677302 remaining
Transfered 10717042 of 13334850 - 2617808 remaining
...
Syncing /system/app/YouTube.apk
Transfered 12817565 of 13334850 - 517285 remaining
Transfered 12883101 of 13334850 - 451749 remaining
Transfered 12948637 of 13334850 - 386213 remaining
Transfered 13014173 of 13334850 - 320677 remaining
Transfered 13079709 of 13334850 - 255141 remaining
Transfered 13145245 of 13334850 - 189605 remaining
Transfered 13210781 of 13334850 - 124069 remaining
Transfered 13276317 of 13334850 - 58533 remaining
Transfered 13334850 of 13334850 - 0 remaining

Thursday, June 24, 2010

Working with the SyncService

I have gotten a few more key parts of the Managed Android Debug Bridge done. The SyncService has some important methods completed. I have been able to successfully pull a single file from a device, and push a single file to a device. Still not working is the ability to push and pull multiple files, but that is next.

Even though this is a port of the DDMS library, I have taken it upon myself to make some needed changes to some components. For example, when pulling a file, here is the original ddmslib code:

public SyncResult pullFile(String remoteFilepath, String localFilename, ISyncProgressMonitor monitor) {
monitor.start(0);
//TODO: use the {@link FileListingService} to get the file size.

SyncResult result = doPullFile(remoteFilepath, localFilename, monitor);
monitor.stop();
return result;
}


Notice the “TODO”, this means that when this method is used to pull a file, there is no “notifications” to the ISyncProgressMonitor since the total bytes is set to 0 in the monitor.start. I took the 5 minutes to actually write the method in the FileListingService that can take a string path, and find the FileEntry object, or it will throw a FileNotFoundException. Once we have the FileEntry, we can determine how much data is going to be transfered. Here is the same method, but in MADB.



public SyncResult PullFile ( String remoteFilepath, String localFilename, ISyncProgressMonitor monitor ) {
if ( monitor == null ) {
throw new ArgumentNullException ( "monitor", "Monitor cannot be null" );
}

long totalWork = 0;
try {
FileListingService fls = new FileListingService ( this.Device );
FileEntry remoteFileEntry = fls.FindEntry ( remoteFilepath );
totalWork = remoteFileEntry.Size;
} catch ( FileNotFoundException ffe ) {
Console.WriteLine ( ffe.ToString ( ) );
Log.w ( "ddms", ffe );
}
monitor.Start ( totalWork );

SyncResult result = DoPullFile ( remoteFilepath, localFilename, monitor );

monitor.Stop ( );
return result;
}


If it fails to find the FileEntry, then it continues on just like the original does. Here is the FindFileEntry method in the FileListingService.



public FileEntry FindFileEntry ( String path ) {
String[] entriesString = path.Split ( new char[] { LinuxPath.DirectorySeparatorChar }, StringSplitOptions.RemoveEmptyEntries );
FileEntry current = this.Root;
foreach ( var pathItem in entriesString ) {
FileEntry[] entries = GetChildren ( current, true, null );
foreach ( var e in entries ) {
if ( String.Compare ( e.Name, pathItem, false ) == 0 ) {
current = e;
continue;
}
}
}
if ( String.Compare ( current.FullPath, path, false ) == 0 ) {
Console.WriteLine ( "returning: {0}", current.FullPath );
return current;
} else {
throw new FileNotFoundException ( String.Format ( "Unable to locate {0}", path ) );
}
}


Another thing that I thought should be modified was the Regex used for the FileListingService. In the original version, it only supports the stock Android ls command. I have modified it to support the stock ls, plus it supports busybox ls as well. This was a selfish act, since it helps me closer to my goal of not requiring busybox in order for Droid Explorer to work.

Wednesday, June 23, 2010

MADB FileListingService

mad-hatter-hatThe Managed Android Debug Bridge is coming along. I did some work on the FileListingService, but messed up the checkin and its now missing some files in the project. This will be fixed tomorrow. The FileListingService is the component that is responsible for ADB request that require file information. At the core, it executes an “ls –l” on the device and parses the return data for the information.

I have made some modifications to the Regex used to parse the return data so that both stock “ls” and “busybox ls” are supported. This means that Droid Explorer will be able to perform some operations on non-rooted devices once MADB is integrated in. The good news is, Droid Explorer + MADB will also be able to detect if busybox is installed on the device. It will also be able to display more detailed information, even more then ADB, when pushing or pulling a file to / from the device.

MADB gives more information to developers when executing remote commands when the file does not exist, or even if an “ls” is attempted on a directory does not exist. It does so by throwing a FileNotFoundException. Otherwise, the specified IShellOutputReceiver will be the way developers will know what is happening in the remote command.

Monday, June 21, 2010

Managed Android Debug Bridge

I have been slowly working on a side project of Droid Explorer called Managed Android Debug Bridge (MADB) and I finally have checked in some code that actually compiles.

MADB is a port of the ddms java library that the ddms uses. This allows communication with the ADB server via TCP and gives more flexibility to developers then calling the adb binary directly. For starters, MADB can request the FrameBuffer, which is what is used to get a screenshot of the device. This is different then pulling /dev/graphics/fb0 because:

  1. that requires root and MADB doesn’t
  2. fb0 contains 2+ frames of data and MADB only gets 1
  3. MADB is faster since its less data to transfer
  4. MADB returns the device screen resolution, ARGB length & offset & bpp. fb0 has none of that information

I have a very small portion of it implemented. Currently, I have the GetFrameBuffer, ExecuteRemoteCommand, Reboot & GetDevices implemented. I’ve actually gone a little further with ExecuteRemoteCommand then the original library, as it checks for the error message that would be displayed if the binary was not found and the method throws a FileNotFoundException instead of just handing the data off to the IShellOutputReceiver.

The goal of MADB is to mimic the functionality that is available via the ddms library and make it available to .NET projects. This project will also be used in Droid Explorer once it has the needed functionality that Droid Explorer requires.

Saturday, June 19, 2010

Issue with Visual Studio 2010 and dark color themes

I have recently moved to a dark color theme for all my code editors. It is much easier on the eyes. But I recently came across an issue that I can’t figure out how to correct and was hoping that someone will have an answer.
When the method info box pops up this is what it looks like
vs2010dark
Now obviously this is not very useful as you can only read “void” and even that is hard to read. I either need to be able to change the background color of the popup window or the color of the text that is displayed in there.
If anyone is interested, the color theme I am using is based on “Coding Instinct” found on StudioStyles.
Here is my modified version, just has some tweaks that were not in it when I started using it, like the line numbers background was white and when you hover a collapsible block it was white, things like that.

Update:
Seems this problem is caused by Visual Studio 2010 Pro Power Tools. It enables the code highlighting in the to the parameter help. Email your congressman to see if we can get a way to fix or disable this.
 
Update 2:
This can be fixed. In the Options->Environment->Fonts and Colors set Signature Help Tooltip Background to a darker color of your choice.

Device Information Extras

madb-xunitMy goal for Managed Android Debug Bridge is not just to port the functionality from DDMS/ADB, but to make it useful to developers.

The original Device in ddms has a dictionary for mount points, but it only has the 3 hard coded m ount points. So if a rooted device, with busybox is connected, then it will pull all the mount points on the system.

 

 

 

This gives some information like this:

rootfs			/			rootfs	ro
tmpfs /dev tmpfs rw
devpts /dev/pts devpts rw
proc /proc proc rw
sysfs /sys sysfs rw
tmpfs /sqlite_stmt_journals tmpfs rw
none /dev/cpuctl cgroup rw
/dev/block/mtdblock3 /system yaffs2 ro
/dev/block/mtdblock5 /data yaffs2 rw
/dev/block/mtdblock4 /cache yaffs2 rw
/dev/block/mmcblk0p2 /sd-ext ext2 rw
/dev/block/vold/179:1 /sdcard vfat rw

Notice how the /system mount is read-only. That is because I also have a method to set the mount point ReadWrite/ReadOnly. Again, this will really only be useful for rooted devices, but it also gives the ability to check what the current “access” of the mount is as well. So Droid Explorer can check if the mount is read-only before doing a file operation, like a delete or create, and it can change the mount to read-write if needed (and if it is "enabled" by the user).


MADB also can transfer files better then regular adb. The ddms library does not handle symlinks, but MADB does. So if you try to copy a symlink, it will resolve it and copy over the original, but still name it what the symlink is.


Pulling multiple files from the device can also easily allow developers to show the total progress for the transfer. For the unit tests, I created a simple ISyncProgressMonitor that writes how many bytes were transferred, out of the total bytes, plus how many bytes are remaining. Here is a sample output, with some data trimmed for space, of a pull of /system/app


Starting Transfer
Syncing /system/app/BugReport.apk
Transfered 23669 of 13334850 - 13311181 remaining
Syncing /system/app/EnhancedGoogleSearchProvider.apk
Transfered 89205 of 13334850 - 13245645 remaining
Transfered 154741 of 13334850 - 13180109 remaining
Transfered 220277 of 13334850 - 13114573 remaining
Transfered 274775 of 13334850 - 13060075 remaining
...
Syncing /system/app/Street.apk
Transfered 10224322 of 13334850 - 3110528 remaining
Transfered 10264332 of 13334850 - 3070518 remaining
Syncing /system/app/Talk.apk
Transfered 10329868 of 13334850 - 3004982 remaining
Transfered 10395404 of 13334850 - 2939446 remaining
Transfered 10460940 of 13334850 - 2873910 remaining
Transfered 10526476 of 13334850 - 2808374 remaining
Transfered 10592012 of 13334850 - 2742838 remaining
Transfered 10657548 of 13334850 - 2677302 remaining
Transfered 10717042 of 13334850 - 2617808 remaining
...
Syncing /system/app/YouTube.apk
Transfered 12817565 of 13334850 - 517285 remaining
Transfered 12883101 of 13334850 - 451749 remaining
Transfered 12948637 of 13334850 - 386213 remaining
Transfered 13014173 of 13334850 - 320677 remaining
Transfered 13079709 of 13334850 - 255141 remaining
Transfered 13145245 of 13334850 - 189605 remaining
Transfered 13210781 of 13334850 - 124069 remaining
Transfered 13276317 of 13334850 - 58533 remaining
Transfered 13334850 of 13334850 - 0 remaining

Working with the SyncService

I have gotten a few more key parts of the Managed Android Debug Bridge done. The SyncService has some important methods completed. I have been able to successfully pull a single file from a device, and push a single file to a device. Still not working is the ability to push and pull multiple files, but that is next.

Even though this is a port of the DDMS library, I have taken it upon myself to make some needed changes to some components. For example, when pulling a file, here is the original ddmslib code:

public SyncResult pullFile(String remoteFilepath, String localFilename, ISyncProgressMonitor monitor) {
monitor.start(0);
//TODO: use the {@link FileListingService} to get the file size.

SyncResult result = doPullFile(remoteFilepath, localFilename, monitor);
monitor.stop();
return result;
}


Notice the “TODO”, this means that when this method is used to pull a file, there is no “notifications” to the ISyncProgressMonitor since the total bytes is set to 0 in the monitor.start. I took the 5 minutes to actually write the method in the FileListingService that can take a string path, and find the FileEntry object, or it will throw a FileNotFoundException. Once we have the FileEntry, we can determine how much data is going to be transfered. Here is the same method, but in MADB.



public SyncResult PullFile ( String remoteFilepath, String localFilename, ISyncProgressMonitor monitor ) {
if ( monitor == null ) {
throw new ArgumentNullException ( "monitor", "Monitor cannot be null" );
}

long totalWork = 0;
try {
FileListingService fls = new FileListingService ( this.Device );
FileEntry remoteFileEntry = fls.FindEntry ( remoteFilepath );
totalWork = remoteFileEntry.Size;
} catch ( FileNotFoundException ffe ) {
Console.WriteLine ( ffe.ToString ( ) );
Log.w ( "ddms", ffe );
}
monitor.Start ( totalWork );

SyncResult result = DoPullFile ( remoteFilepath, localFilename, monitor );

monitor.Stop ( );
return result;
}


If it fails to find the FileEntry, then it continues on just like the original does. Here is the FindFileEntry method in the FileListingService.



public FileEntry FindFileEntry ( String path ) {
String[] entriesString = path.Split ( new char[] { LinuxPath.DirectorySeparatorChar }, StringSplitOptions.RemoveEmptyEntries );
FileEntry current = this.Root;
foreach ( var pathItem in entriesString ) {
FileEntry[] entries = GetChildren ( current, true, null );
foreach ( var e in entries ) {
if ( String.Compare ( e.Name, pathItem, false ) == 0 ) {
current = e;
continue;
}
}
}
if ( String.Compare ( current.FullPath, path, false ) == 0 ) {
Console.WriteLine ( "returning: {0}", current.FullPath );
return current;
} else {
throw new FileNotFoundException ( String.Format ( "Unable to locate {0}", path ) );
}
}


Another thing that I thought should be modified was the Regex used for the FileListingService. In the original version, it only supports the stock Android ls command. I have modified it to support the stock ls, plus it supports busybox ls as well. This was a selfish act, since it helps me closer to my goal of not requiring busybox in order for Droid Explorer to work.

MADB FileListingService

mad-hatter-hatThe Managed Android Debug Bridge is coming along. I did some work on the FileListingService, but messed up the checkin and its now missing some files in the project. This will be fixed tomorrow. The FileListingService is the component that is responsible for ADB request that require file information. At the core, it executes an “ls –l” on the device and parses the return data for the information.

I have made some modifications to the Regex used to parse the return data so that both stock “ls” and “busybox ls” are supported. This means that Droid Explorer will be able to perform some operations on non-rooted devices once MADB is integrated in. The good news is, Droid Explorer + MADB will also be able to detect if busybox is installed on the device. It will also be able to display more detailed information, even more then ADB, when pushing or pulling a file to / from the device.

MADB gives more information to developers when executing remote commands when the file does not exist, or even if an “ls” is attempted on a directory does not exist. It does so by throwing a FileNotFoundException. Otherwise, the specified IShellOutputReceiver will be the way developers will know what is happening in the remote command.

Managed Android Debug Bridge

I have been slowly working on a side project of Droid Explorer called Managed Android Debug Bridge (MADB) and I finally have checked in some code that actually compiles.

MADB is a port of the ddms java library that the ddms uses. This allows communication with the ADB server via TCP and gives more flexibility to developers then calling the adb binary directly. For starters, MADB can request the FrameBuffer, which is what is used to get a screenshot of the device. This is different then pulling /dev/graphics/fb0 because:

  1. that requires root and MADB doesn’t
  2. fb0 contains 2+ frames of data and MADB only gets 1
  3. MADB is faster since its less data to transfer
  4. MADB returns the device screen resolution, ARGB length & offset & bpp. fb0 has none of that information

I have a very small portion of it implemented. Currently, I have the GetFrameBuffer, ExecuteRemoteCommand, Reboot & GetDevices implemented. I’ve actually gone a little further with ExecuteRemoteCommand then the original library, as it checks for the error message that would be displayed if the binary was not found and the method throws a FileNotFoundException instead of just handing the data off to the IShellOutputReceiver.

The goal of MADB is to mimic the functionality that is available via the ddms library and make it available to .NET projects. This project will also be used in Droid Explorer once it has the needed functionality that Droid Explorer requires.

Issue with Visual Studio 2010 and dark color themes

I have recently moved to a dark color theme for all my code editors. It is much easier on the eyes. But I recently came across an issue that I can’t figure out how to correct and was hoping that someone will have an answer.
When the method info box pops up this is what it looks like
vs2010dark
Now obviously this is not very useful as you can only read “void” and even that is hard to read. I either need to be able to change the background color of the popup window or the color of the text that is displayed in there.
If anyone is interested, the color theme I am using is based on “Coding Instinct” found on StudioStyles.
Here is my modified version, just has some tweaks that were not in it when I started using it, like the line numbers background was white and when you hover a collapsible block it was white, things like that.

Update:
Seems this problem is caused by Visual Studio 2010 Pro Power Tools. It enables the code highlighting in the to the parameter help. Email your congressman to see if we can get a way to fix or disable this.
 
Update 2:
This can be fixed. In the Options->Environment->Fonts and Colors set Signature Help Tooltip Background to a darker color of your choice.