/ Chef

lazy { is good }

Being lazy is generally not considered a great virtue. However, sometimes it can be good. Such is the case with the lazy{}, the delayed evaluation provided for us to use in Chef code. As explained in the this section of the "common functionality" doc, the lazy evaluation property is useful when the value of a property in a resource can't be known until the execution phase of the chef-client run[1].

Lazy isn't the only delayed evaluation block available, and its cousins are probably more familiar: not_if, only_if, and ruby_block. If you have had any chance to write Chef code, at least two of those are very familiar. Thinking about how the not/only_if code works should make the lazy evaluation a little more clear. For instance, consider the situation where you have a resource that should only take action if a certain file is present on the machine, but that file is actually created earlier in the execution phase of the chef-client run:

file '/tmp/gotime'

execute 'do something if its go time' do
  command '/usr/local/bin/doit.sh'
  only_if { ::File.exist? '/tmp/gotime' }
end

So then, we see the value of delayed evaluation of ruby code within the Chef run. If the not_if above were to be evaluated during the compile phase, our execute resource would not be idempotent, and nobody wants that.

Lazy differs from the other delayed evaluation methods, in that it isn't designed to guard a resource idempotentcy (not_if / only_if), or to be a resource unto itself (ruby_block). It is meant to augment a resource, allowing it rely on data that is not available during the compile phase of the chef-client run. While not necessary at all, it is often used in conjunction with node.run_state, which is a way of collecting and passing data between resources, and used during the execution phase of the chef-client run. Since node.run_state data isn't available at the compile phase, a delayed evaluation method is required.

I recently had cause to use lazy fairly extensively in a cookbook. Before that, I had rarely used it, but found it extremely helpful in keeping the flow of this cookbook consistent and quick. In a nutshell, I was writing a cookbook to configure some IBM Power hardware via an HMC. The cookbook uses SSH gather information and execute configuration steps (more like Ansible might), and the HMC is not known for it's blazing fast speeds. Given that, I wanted to collect all the data I needed for idempotentcy and decision making at the beginning of the recipe(s) and store them in node.run_state data for use in lazy evaluation. For instance, to get all the installed card information, I call a custom resource and assign the values to `node.run_state['cards']:

# Get frame list
hmc 'managed-systems' do
  hostname hmc_hostname
  username hmc_username
  password hmc_password
  action :get_managed_systems
end

# Get card slot information
hmc 'cards' do
  hostname hmc_hostname
  username hmc_username
  password hmc_password
  action :get_card_info
  frame lazy { 
    node[:orig_name] if node.run_state['managed-systems'].include? node[:orig_name]
    node[:new_name] if node.run_state['managed-systems'].include? node[:new_name]
  }
end

Remembering that each of these actions is an SSH session reaching out to the HMC in question, I first gather the list of managed systems known to the HMC. This list is stored to node.run_state['managed-systems'] as designed in the custom resource hmc :get_managed_systems action. The :get_card_info of the same resource is then called to find the installed card information. However, part of the configuration defined with in this cookbook is also to rename the managed system from it's shipped name. Since a recipe should be able to run multiple times and be idempotent, if the managed system was already renamed, then we have to come up with a way to determine the hardware to gather card info by name, which is the first example of using lazy from above. The frame property is the name of the managed system. Since I cannot reasonably determine if it has been renamed already, I simply use lazy to delay the evaluation of the value sent to the property, and use list of managed systems to decide which one needs to be used here (we assume uniqueness for both the desired and shipped name). If the hardware had been renamed, the desired name will be passed into the query, and the shipped name if not.

That is a good use, but as complex as it can get. Here is a snippet from the same cookbook, which uses lazy in a bit more complex way.

node[:vios].each do |vio,viodefs|
  hmc "#{viodefs[:lpar_name]} profile" do
    hostname hmc_hostname
    ...
    raid_card lazy { node.run_state['cards'][viodefs[:raid]]['drc_index'] }
    fc_adapter lazy { node.run_state['cards'][viodefs[:fc_adapter]]['drc_index'] }
    sriov_cards lazy {
      # Create hash of SR-IOV cards
      sr_cards = Mash.new
      viodefs[:sriov_adapter].each do |srcard,ports|
        Chef::Log.warn("#{srcard} ---- #{ports}")
        sr_cards[node.run_state['sriov'][srcard]['adapter_id']] = ports
      end

      sr_cards
    }
    not_if { node.run_state['lpars_status'].include? viodefs[:lpar_name] }
  end
end

Here I am building profiles, basically the virtual machine definition, to install VIOS. As part of the profile, the adapter unique identifiers need to be passed in to compile the command that will be sent to the HMC. A JSON file is passed in as part of the chef-client zero run, which contains the desired state of the VIO profiles, including the hardware location codes (not the same as the unique identifies required by the command). With the data collections done and placed into node.run_state, the identifiers can be found, as the data is keyed by the hardware location codes. Basically node.run_state['cards'] and node.run_state['sriov'] are hashes I need to search with values from the viodefs derrived out of the JSON file previously mentioned. Clear as mud? Yeah, sometimes dealing with IBM type stuff isn't super easy.

So anyway, the cool part is there in sriov_cards. First, given part of the JSON file that describes the SRIOV card/ports required for one of the VIOS:

{
  "sriov_cards": {
    "hardware_loc_1": [0, 1, 2],
    "hardware_loc_2": [0, 1, 2]
  }
}

Also, a custom resource like above, collecting the SRIOV card information from the entire managed system, and storing the hardware location keyed data in node.run_state['sriov'], and we have the building blocks for:

    sriov_cards lazy {
      # Create hash of SR-IOV cards
      sr_cards = Mash.new
      viodefs[:sriov_adapter].each do |srcard,ports|
        sr_cards[node.run_state['sriov'][srcard]['adapter_id']] = ports
      end

      sr_cards
    }

Once that code is evaluated in the execution phase, it will send a hash to the sriov_cards property of the custom resource that woudl look something like:

{
  "sriov_cards": {
    "unique_id": [0, 1, 2],
    "unique_id": [0, 1, 2]
  }
}

That seems simple from a human perspective, however that information was simply not available at compile time, when the code was being ordered and in-memory representations of the resources are being stored. Meaning, some cool and interesting things are happening.

The first is that the lazy block is just that, a standard block of code, in which any logic needed can be coded, and the value exiting lazy is declared just like any standard ruby function (line #8). However, any reference to variables, attributes, what have you, are left as references and not replaced by their values. They are left, as the name implies, to be evaluted later.

Secondly, it's like querying the future. Remember, at compile time node.run_state['sriov'] isn't even a hash that exists, yet we are asking for the value of 'adapter_id' from, in this case, two of its keys. When execution time comes, the code is evaluated, and the values I requested are dutifully returned, and used to create the sr_cards hash which is then passed into the hmc custom resources property sriov_cards. So, in essence, we are creating code for things that don't exist, but magically appear later, and are properly referenced, because the evaluation of the logic is delayed. On the surface, that sounds fairly simple. But when you look at more complex applications, the things that it can allow to be placed into recipes is pretty much the limit of the imagination of the coder.

But wait, there's more! Lazy can also be used for determining things like default values for attributes inside custom resources. Imagine a situation where you have a custom resource to execute some code contained in a file. The location of that file can be passed into the resource via some property, say script_location. Also imagine that the location of that file may not be passed into the resource, so a default location has to be assigned, but the location could be variable for some reason. Something like this is what lazy is meant for:

resource _name :cool_thing

property :something, String, name_property: true
...
property :script_location, String, default lazy { find_script }

action :run do
  execute "execute #{something}" do
    command script_location
  end
end

def find_script
  %w(/usr/local/bin/loc1 /usr/local/bin/loc2 /opt/thing/loc3).each do |possible_location|
    possible_location if ::File.exist? possible_location
  end
end

In the example above, the script_location can be passed in via the resource when declared, or if it is not, a function will be lazily called to determine what it should be. This allows for taking platform differences into account without having to create different custom resource for each. It could also allow for the code being executed to be created by the same chef-client run, where it possibly didn't exist before.

In closing, learning how to effectively use lazy delayed evaluation can make your life easier, and help you to see into the future :)


  1. chef-client runs are actually achieved with a two pass model. This will explain it, if that is something new to you. ↩︎