Running in the name of…

In Active Directory, Chef, DevOps, Powershell, Race Against the Machine on February 14, 2015 at 22:28

During my last post I briefly discussed the issues related to the execution context of the chef-client when developing and testing cookbooks. To summarize: on a windows machine the chef-client runs under the context of the local system account, an account which basically is a local admin but has almost no permissions on other Active Directory joined/secured objects ( such as other machines, Active Directory itself or file shares).

You can always mimic this behaviour by just adding a cookbook to a test node’s runlist and restarting the chef-client service of course. Or create a scheduled task which runs chef-solo or zero. However that leaves you without the direct output from an interactively started chef-run. Also I can’t really see how you could add something like that to a vagrant setup.

Luckily – the need to run stuff under the local system account is not new. Quite some time ago the windows tool legend Mark Russinovich developed a tool which also you to, amoungst other things, to start a process as the local system account. In order to do that it temporarily installs a windows service called PSEXESVC which is used to start a process ( eg. cmd.exe) as the local system account.

The command itself is very simple:

somepath:\psexec /s cmd.exe

You can run this from another command window, or Powershell – doesn’t really matter. Once you do it starts another instance of cmd.exe and everything executed from that instance will be run as the local system account. For instance :


Will return:

nt authority\system

From there you can start chef-client just as you normally would – however you will find out that if you access anything outside of the node that access will be denied – unless the permissions on the resource are set to include the computer account object ( which includes Everyone and Authenticated Users btw). Also the output of the chef-client is a little bit different – more like the output that’s written to the chef log when you run the client as a service.

I’ve tried hacking this into the batch files chef uses to start stuff ( so instead of calling ruby.exe directly I would first open a command console as local system when running chef zero) but that didn’t seem to work – I guess that has something to with the difference between a real interactive session and a session as started by psexec. But I’m sure I’ll find some way to work around that at some point.

But for now – if you really want to know if your cookbook will run as you’d expect in a Windows domain environment – use psexec manually at least once in order to identify any permission issues.


P.S – there is one other fun way to run stuff as local system – replacing the ease of access executable on a box ;-) For more details see Guy’s site.

Steamy Windows – 3 things no one told you before you started with Chef on Windows

In Active Directory, Chef, DevOps on February 5, 2015 at 21:42


Steamy windows
Zero visibility”

1. Escape


Escape characters in Ruby ( and therefore in Chef) start with a backslash. Nothing wrong with that.But it will mess up your paths when you are working on windows – because in the windows world our paths are made out of backslashes ( a nice article on the history or back slashes in windows can be found here.

Online sources and docs point you towards helpers and functions to fix this. However in my experience the best way to deal with this is just escaping them properly. And since Windows has no issues with double backslashes instead of single back slashes you could – as a precaution – just use double back slashes in all your attribute assignments.

This does mean however that an UNC path needs double escaping, e.g. \\someserver\somepath. More about UNC paths and fileshares later.


2. Thou Powershall not


Chef supports Powershell.

Or well actually it provides you with a couple of ways to run powershell from a recipe. Powershell_script will create a temporary script on the file system and run that and Powershell_out actually starts a powershell process and runs the code you provide.

However is important to realize that while you can use recipe variables such as node attributes in your powershell script – you can’t actually pass back the objects powershell returns. Powershell_script just returns the exit code and powershell_out captures all return data – but in a string – which is perfectly suitable for error handling or reporting but it denies you access to one of the big advantages of powershell, the fact that you can gather all sorts of information about your environment and use that in your script’s decision making logic without having to parse the ouput and put it into some sort of data structure.

Par example – there is no really added value to using the AD cmdlets over plain old dsquery because if you want to use data it returns you’ll have to parse the returned strings with some sort of fancy regex and what not. And in all honesty powershell is much slower then most cmdline based tools – especially when they are RPC rather then WMI based.

This is by no means a design flaw or anything – the whole idea behind Chef is that when you start a run you already have the necessary environment data, either from Ohai or from loading up your defined resources.

So either use equivalent ruby code or fall back to the old cmdline binaries ( where I personally prefer using ruby code )

3. Execution Context


This in itself has actually very little to do with Chef. If you want to run something on a box you’ll need to do that under some sort of security context which determines what you can and what you cannot do. Before you get access to that context you will need to authenticate.

If you want to automate a whole set of systems there you really two things you can do :

1. Run commands and execute them remotely on a system with some sort of network shell ( an example could be running a powershell remoting session from an admin machine to a set of target machines).

2. Use a service or a scheduled task to run tasks locally on each machine.

Chef and quite a few other config management tools use the second option. And this works really well if you only need to change things on that particular node. You can just use the Local System account – no password needed and it has all the permissions you might need. No it no suprise that this is the account the chef-client uses to do it’s regular runs.

But what if you’d need to change a setting outside of the node/machine? Like creating a DNS entry, or a SPN or use the template resource on a CIFS\SMB Share? The local system account ( even though it has some permissions when the machine is domain joined – such as being a member authenticated users) cannot access either of these resources.  And what further complicates this is, is that in the “chef world” – testing and development is generally done by running chef with some sort of “real user” ( either the person logged on the machine or the vagrant user when you are using vagrant). So you won’t notice that you have a problem until you actually deploy your cookbook.

There are three ways to solve this :

1. Use chef_shellout in your recipes.It supports running a commmand as another user( basically what it does is spawn a process). Or when you are dealing with network shares, use the mount provider. It allows you to mount a share as another user as well. All actions you perform after mounting the share will executed as that user. Even though it might seem tempting to use shell_out to call Powershell remember that at the moment this has very little added value.

The powershell_out mixin btw strangly enough has no option to run as another user (yet).

2. To the outside world ( inside your domain anyway) – the Local System account will perform actions outside of the host as the AD computer account. So if you try to reach a share from Chef on let’s say windowsbox1 while Chef-client is running as the local system account ( which is the default and as far as I’ve seen the only recommended configuration) you will see a logon request from contoso\windowsbox1. You could of course just grant this machine account permission to your share ( either directly our through some group). This may sound crazy – but there are other situations were this is perfectly normal ( such as Microsoft Clustering, Kerberos constrained delegation in TMG,SCCM and some part of MS Exchange).

One caveat : you really have to have a working Kerberos authentication setup – because authenticating as a computer account won’t work over NTLM.

3. Bypass the windows security context all together. Easier said then done most of the time but some of the infrastructures services actually have a network interface or API ( LDAP, SharePoint). This will enable you to define a bind account or authenticate through web services and only for that specific task or service.

And there might be a fourth – running chef-client under a service account. I haven’t really seen anyone do this though.

In my next post I will try to the pro’s and con’s of these three approaches and show a simple and easy way to simulate this real world limitation when you are developing your cookbooks.

Take a load off Fannie – Chefdk, Rundll32.exe and constant cpu load

In Chef, DevOps, Packer, Vagrant, Windows 8.1 on February 4, 2015 at 21:59


First of all – quite some time ago I decided to move my blog to a new platform, a new server and a new domain. But for some reason that just didn’t work out. So I’m back here.

Luckily the personal change that motivated me to move my blog did work out – I’m still very happily employed at Schuberg Philis.

Secondly, after some hesitation, I’ve fully embraced both Chef and Ruby and quite honestly can’t quite imagine doing things any other way right now.

However even in a perfectly automated and continuously tested environment stuff still breaks. And  working with Chef and related technologies can be daunting – not in the least due to the cultural differences between a classic Windows environment and the Linux world where things like versioned configuration management and configuration mainly based on files has been commonplace for quite some time.

But that is not the topic of this particular post. When I was setting up my new Windows 8.1 laptop I ran into an issue after installing my development tools such as Vagrant, Chefdk and Github for Windows. After a while I noticed that my fan kept blowing and task manager showed that there was an average 25% cpu load ( so basically 100% on one logical core). The process consuming the core was rundll32.exe. I fired up process explorer and found out that rundll32 was loading aeinv.dll, with the parameter: UpdateSoftwareInventory. This pointed towards the Application Experience service.

The Application Experience service is basically the service that helps you when you run a program that has compatibility issues:




I disabled the service however that did not solve the problem. Every now and then the process would turn up again. So I checked some blogs and forum posts and found out that there a couple of scheduled tasks that are enabled and run even if the service is disabled and stopped. I triggered each service and found out three of them spawned the rundll32 process. So I disabled all three of them – problem solved.

Now how is this related to Chefdk? Well procmon showed that all the rundll32 process was doing was iterating over the c:\opscode folder and its children. Why? I’m not sure – at first I thought it might be the sheer number of files ( 41k files on my box to be precise) – but I’ve seen other users question that.

Anyway if you are faced with this problem – the solution is actually pretty simple and painless.

P.S. – was so kind to point out that only one of the three tasks actually causes this problem – the ProgramDataUpdater task. To me that name sounded  familiar – I’ve seen that task pop up a couple of times in VMware View and Citrix XenDesktop optimization guides as well ( one of them by Microsoft).

I’ve also seen some people associate this with MSI file checking behaviour  – maybe that’s something to check at some point as well.



Get every new post delivered to your Inbox.

Join 97 other followers