Sunday, June 12, 2016

Installing Git on a Jenkins Windows Slave

Recently working on getting a Jenkins Windows build slave up and running, I noticed that the builds would hang, and timeout. The build would get stuck during the clone of the repository. Furthermore, if the build tried to run again, it would error out while trying to clean up the workspace.

We are using Chef and the Jenkins Cookbook to set up the Windows slave. Git for Windows is installed via Chocolatey. After a couple days of trial and error, here is what I found, and how I fixed it.

 

The latest versions of Git for Windows installer, we are installing 2.8.4, installs the Credential Manager by default. I have been unable to find a way to disable this option during the install, if you know a way, I would love to hear the solution to that.

(Here is a screenshot I found of the option, though it is from the 2.7.4 installer)


The credential manager stores your git user/password for your repositories, so you don’t have to enter them every time. Well, we are using Jenkins, and Jenkins does not get mad about having to enter the credentials every time. Also, the credential manager pops up this dialog when you try to clone a repository that requires authentication

 

When this pops up, the Jenkins build executor is stuck because it does not know how to enter input into this dialog.

 

We need to tell git that we do no want to use this credential helper, since we have no way to prevent it from being installed. To do this, with chef, I created a file in files/default called base.gitconfig.

[core]
	autocrlf = true
	askpass = true

Next, in the recipe, I put the file on the system in c:\program files\git\mingw64\etc\ named gitconfig.

cookbook_file "C:/Program Files/Git/mingw64/etc/gitconfig" do
  source 'base.gitconfig'
  owner node['jenkins']['os']['windows']['user']
  group node['jenkins']['os']['windows']['group']
end
Now when the Windows slave attempts to do a git clone, it no longer will use the credential manager and pop up the dialog, and everything works flawlessly.

Thursday, April 14, 2016

Change Colors in PowerShell Console

I have my powershell console window have a black background and slightly transparent. The default colors here work for most things, except errors. Errors were hard to read, especially when projecting to a secondary monitor. I found on the web that you can change the foreground color for powershell errors by setting $host.privatedata.ErrorForegroundColor. But the problem with these solutions that I found was that they just said “from a PS prompt, set the color”. I wanted something a little more permanent.

What I came up with was to set the values that I want in my PowerShell profile file that is located at [Environment]::GetFolderPath("MyDocuments")/WindowsPowerShell/Microsoft.PowerShell_profile.ps1.

Here is what I put in mine:

$host.privatedata.ErrorForegroundColor = 'Magenta';
$host.privatedata.ErrorBackgroundColor = 'Black';
$host.privatedata.WarningForegroundColor = 'Yellow';
$host.privatedata.WarningBackgroundColor = 'Black';
$host.privatedata.DebugForegroundColor = 'DarkGray';
$host.privatedata.DebugBackgroundColor = 'Black';
$host.privatedata.VerboseForegroundColor = 'White';
$host.privatedata.VerboseBackgroundColor = 'Black';
$host.privatedata.ProgressForegroundColor = 'DarkBlue';
$host.privatedata.ProgressBackgroundColor = 'DarkCyan';

 

image

Now my errors are readable.

To see what values you can set them to just execute: [system.consolecolor]::GetNames("consolecolor").

Tuesday, April 12, 2016

Running kitchen converge on Windows 2012r2 image on OpenStack

I have been working with my team to get Windows 2012r2 image chef cookbook to converge in an OpenStack environment. We have run in to a lot of issues, and I am confident that some of those were because of my lack of experience with chef/windows. One of the big issues that we encountered was kitchen unable to WINRM in to the box. It would just keep throwing [winrm] HTTPClient::ConnectionTimeoutException. We noticed that if someone manually logs on to the machine, and they are presented with the User must change password for the account. After they enter the password, they are prompted with Choose the type of network for the machine.

This had me thinking. If the user account is set to require the password to be changed, Windows will not allow anyone to remotely logon to the machine with that account. So I tested this theory. I spun up the Windows2012r2 image in OpenStack and attempted to connect with Enter-PSSession. I was getting an error that the connection could not be established. Specifically:

WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, 
and that a firewall exception for the WinRM service is enabled and allows access from this computer. By default, the WinRM firewall
exception for public profiles limits access to remote computers within the same local subnet.

This lead me to determine that because the network type has not been selected, the second thing the user is presented with after manually logging in, WinRM is not letting the connection happen. I then figured that because this machine has CloudInit, there has to be a way to set the network type to be private. If this machine was added to the Domain, this would not be an issue, but this is a ephemeral machine just to test the changes to the cookbook and verify it works so it is not on the Domain. After some research I came across Setting Network Location to Private blog post by Vladimir Averkin. This post has a block of PowerShell that enumerates all networks, post-Vista machines that are not joined to a Domain, and sets them to Private.

# Skip network location setting for pre-Vista operating systems
if([environment]::OSVersion.version.Major -lt 6) { return } # Skip network location setting if local machine is joined to a domain.
if(1,3,4,5 -contains (Get-WmiObject win32_computersystem).DomainRole) { return } # Get network connections
$networkListManager = [Activator]::CreateInstance([Type]::GetTypeFromCLSID([Guid]"{DCB00C01-570F-4A9B-8D69-199FDBA5723B}"))
$connections = $networkListManager.GetNetworkConnections() # Set network location to Private for all networks
$connections | % {$_.GetNetwork().SetCategory(1)}

Perfect!

I launched a new instance on OpenStack, this time in the Post-Creation I added the script above, making sure to specify #ps1_sysnative at the top to let cloudinit know that we are running some PowerShell. Then waited…

Fired up PowerShell and ran Enter-PSSession... Failed, Access is denied. This has to be the issue with the user being prompted to change the password. Went back into OpenStack, launched another instance. This time at the end of the above script, I added:

net user KitchenCIUser KitchenCIUserPassword

Then I waited again… I started running Test-WSMan until it came back successful. Then I ran Enter-PSSession one more time. SUCCESS! I was now connected to the machine via PSRemoting.

Now one last hurdle, how do I specify the CloudInit script with the .kitchen.yml. Well, as it turns out, the kitchen-openstack driver supports this.

driver_config:
  user_data: cloud-init.txt

Then just have to create the cloud-init.txt file in the same path as the .kitchen.yml. I saved the changes, committed to repository and watched as jenkins kicked off the build. Oh, and I waited…

Success! Kitchen was able to use WinRM and converge the box, now on to the next problem that I am sure we will encounter.

Saturday, April 2, 2016

Droid Explorer working on Non-Rooted Device

I have started doing some work on removing the dependency on rooted device, specifically, busybox. This is no where near complete, and is buggy, at best. As you can see from the screenshot, you can see most of the folders, but if you try to go in to some of them, you will not see any files. This is my Nexus 6P, which is not rooted, connected to Droid Explorer. I do not have busybox or anything else special installed.

no-root

It doesn’t look like much, but this is a big change to the code base. I have been removing code from the Core.CommandRunner, the code that handles 99% of the communication between your computer and your device. Instead it is making use of madb, which minimizes the need for adb, and only uses it to act as the adb server. While the old code used adb as the server, but also depended on the adb client to do the communication to the device. Madb is the client, and it does all the communication to the adb server.

 

No binaries available yet, as I stated, this is very early. The code will be available on the Droid Explorer Github soon.

Saturday, February 27, 2016

VirtualBox VT-X/AMD-V Hardware Acceleration Error

I was getting this error about VT-X/AMD-V Hardware Acceleration not being available on my system and was very confused because I had enabled it in the BIOS and still got the error. Today I decided to do a little research on it. I had remembered in the past reading about Hyper-V causing issues with other virtualization software. So I did some checking on my system, because I know that Hyper-V is enabled/installed when you install the Android Emulator with Visual Studio. I checked if the Hyper-V feature was enabled, and sure enough, it was. I disabled Hyper-V and rebooted my system. Fired up VirualBox and loaded up the VM. Success.

To disable Hyper-V

  • Go to Control Panel –> Programs and Features
  • Click Turn Windows features on or off
  • Uncheck Hyper-V
  • Click OK
  • Restart

Installing Git on a Jenkins Windows Slave

Recently working on getting a Jenkins Windows build slave up and running, I noticed that the builds would hang, and timeout. The build would get stuck during the clone of the repository. Furthermore, if the build tried to run again, it would error out while trying to clean up the workspace.

We are using Chef and the Jenkins Cookbook to set up the Windows slave. Git for Windows is installed via Chocolatey. After a couple days of trial and error, here is what I found, and how I fixed it.

 

The latest versions of Git for Windows installer, we are installing 2.8.4, installs the Credential Manager by default. I have been unable to find a way to disable this option during the install, if you know a way, I would love to hear the solution to that.

(Here is a screenshot I found of the option, though it is from the 2.7.4 installer)


The credential manager stores your git user/password for your repositories, so you don’t have to enter them every time. Well, we are using Jenkins, and Jenkins does not get mad about having to enter the credentials every time. Also, the credential manager pops up this dialog when you try to clone a repository that requires authentication

 

When this pops up, the Jenkins build executor is stuck because it does not know how to enter input into this dialog.

 

We need to tell git that we do no want to use this credential helper, since we have no way to prevent it from being installed. To do this, with chef, I created a file in files/default called base.gitconfig.

[core]
	autocrlf = true
	askpass = true

Next, in the recipe, I put the file on the system in c:\program files\git\mingw64\etc\ named gitconfig.

cookbook_file "C:/Program Files/Git/mingw64/etc/gitconfig" do
  source 'base.gitconfig'
  owner node['jenkins']['os']['windows']['user']
  group node['jenkins']['os']['windows']['group']
end
Now when the Windows slave attempts to do a git clone, it no longer will use the credential manager and pop up the dialog, and everything works flawlessly.

Change Colors in PowerShell Console

I have my powershell console window have a black background and slightly transparent. The default colors here work for most things, except errors. Errors were hard to read, especially when projecting to a secondary monitor. I found on the web that you can change the foreground color for powershell errors by setting $host.privatedata.ErrorForegroundColor. But the problem with these solutions that I found was that they just said “from a PS prompt, set the color”. I wanted something a little more permanent.

What I came up with was to set the values that I want in my PowerShell profile file that is located at [Environment]::GetFolderPath("MyDocuments")/WindowsPowerShell/Microsoft.PowerShell_profile.ps1.

Here is what I put in mine:

$host.privatedata.ErrorForegroundColor = 'Magenta';
$host.privatedata.ErrorBackgroundColor = 'Black';
$host.privatedata.WarningForegroundColor = 'Yellow';
$host.privatedata.WarningBackgroundColor = 'Black';
$host.privatedata.DebugForegroundColor = 'DarkGray';
$host.privatedata.DebugBackgroundColor = 'Black';
$host.privatedata.VerboseForegroundColor = 'White';
$host.privatedata.VerboseBackgroundColor = 'Black';
$host.privatedata.ProgressForegroundColor = 'DarkBlue';
$host.privatedata.ProgressBackgroundColor = 'DarkCyan';

 

image

Now my errors are readable.

To see what values you can set them to just execute: [system.consolecolor]::GetNames("consolecolor").

Running kitchen converge on Windows 2012r2 image on OpenStack

I have been working with my team to get Windows 2012r2 image chef cookbook to converge in an OpenStack environment. We have run in to a lot of issues, and I am confident that some of those were because of my lack of experience with chef/windows. One of the big issues that we encountered was kitchen unable to WINRM in to the box. It would just keep throwing [winrm] HTTPClient::ConnectionTimeoutException. We noticed that if someone manually logs on to the machine, and they are presented with the User must change password for the account. After they enter the password, they are prompted with Choose the type of network for the machine.

This had me thinking. If the user account is set to require the password to be changed, Windows will not allow anyone to remotely logon to the machine with that account. So I tested this theory. I spun up the Windows2012r2 image in OpenStack and attempted to connect with Enter-PSSession. I was getting an error that the connection could not be established. Specifically:

WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, 
and that a firewall exception for the WinRM service is enabled and allows access from this computer. By default, the WinRM firewall
exception for public profiles limits access to remote computers within the same local subnet.

This lead me to determine that because the network type has not been selected, the second thing the user is presented with after manually logging in, WinRM is not letting the connection happen. I then figured that because this machine has CloudInit, there has to be a way to set the network type to be private. If this machine was added to the Domain, this would not be an issue, but this is a ephemeral machine just to test the changes to the cookbook and verify it works so it is not on the Domain. After some research I came across Setting Network Location to Private blog post by Vladimir Averkin. This post has a block of PowerShell that enumerates all networks, post-Vista machines that are not joined to a Domain, and sets them to Private.

# Skip network location setting for pre-Vista operating systems
if([environment]::OSVersion.version.Major -lt 6) { return } # Skip network location setting if local machine is joined to a domain.
if(1,3,4,5 -contains (Get-WmiObject win32_computersystem).DomainRole) { return } # Get network connections
$networkListManager = [Activator]::CreateInstance([Type]::GetTypeFromCLSID([Guid]"{DCB00C01-570F-4A9B-8D69-199FDBA5723B}"))
$connections = $networkListManager.GetNetworkConnections() # Set network location to Private for all networks
$connections | % {$_.GetNetwork().SetCategory(1)}

Perfect!

I launched a new instance on OpenStack, this time in the Post-Creation I added the script above, making sure to specify #ps1_sysnative at the top to let cloudinit know that we are running some PowerShell. Then waited…

Fired up PowerShell and ran Enter-PSSession... Failed, Access is denied. This has to be the issue with the user being prompted to change the password. Went back into OpenStack, launched another instance. This time at the end of the above script, I added:

net user KitchenCIUser KitchenCIUserPassword

Then I waited again… I started running Test-WSMan until it came back successful. Then I ran Enter-PSSession one more time. SUCCESS! I was now connected to the machine via PSRemoting.

Now one last hurdle, how do I specify the CloudInit script with the .kitchen.yml. Well, as it turns out, the kitchen-openstack driver supports this.

driver_config:
  user_data: cloud-init.txt

Then just have to create the cloud-init.txt file in the same path as the .kitchen.yml. I saved the changes, committed to repository and watched as jenkins kicked off the build. Oh, and I waited…

Success! Kitchen was able to use WinRM and converge the box, now on to the next problem that I am sure we will encounter.

Droid Explorer working on Non-Rooted Device

I have started doing some work on removing the dependency on rooted device, specifically, busybox. This is no where near complete, and is buggy, at best. As you can see from the screenshot, you can see most of the folders, but if you try to go in to some of them, you will not see any files. This is my Nexus 6P, which is not rooted, connected to Droid Explorer. I do not have busybox or anything else special installed.

no-root

It doesn’t look like much, but this is a big change to the code base. I have been removing code from the Core.CommandRunner, the code that handles 99% of the communication between your computer and your device. Instead it is making use of madb, which minimizes the need for adb, and only uses it to act as the adb server. While the old code used adb as the server, but also depended on the adb client to do the communication to the device. Madb is the client, and it does all the communication to the adb server.

 

No binaries available yet, as I stated, this is very early. The code will be available on the Droid Explorer Github soon.

VirtualBox VT-X/AMD-V Hardware Acceleration Error

I was getting this error about VT-X/AMD-V Hardware Acceleration not being available on my system and was very confused because I had enabled it in the BIOS and still got the error. Today I decided to do a little research on it. I had remembered in the past reading about Hyper-V causing issues with other virtualization software. So I did some checking on my system, because I know that Hyper-V is enabled/installed when you install the Android Emulator with Visual Studio. I checked if the Hyper-V feature was enabled, and sure enough, it was. I disabled Hyper-V and rebooted my system. Fired up VirualBox and loaded up the VM. Success.

To disable Hyper-V

  • Go to Control Panel –> Programs and Features
  • Click Turn Windows features on or off
  • Uncheck Hyper-V
  • Click OK
  • Restart