Templates.yml key options use case ?!

Pipeline Integrations
,
1

Hi,

I am trying to make sense of the key options inside the templates.yml file.

The option shotgun_entity_type: seems to be a promising solution to the following problem:

Assume the asset field is allowed to contain underscores ‘_’ in it’s name, which can be problematic with the template validation.

Suppose this template definition and it’s evaluation:

>>> my_template.definition
>>> "{Asset}_{shader}_v{version}"
>>> my_template.validate_and_get_fields("my_test_asset_blue_v001")
>>> {"Asset" : "my_test_asset", "Shader" : "blue", "version" : 1 }
# this is correct. The asset "my_test_asset" exists and so does the shader "blue"

We now want to detect files that are not matching this template, with just using the template itself.

>>> my_template.validate_and_get_fields("my_test_asset_blue_some_unpredictableUnderScoreNaming_v001")
>>> {"Asset" : "my_test_asset_blue_some", "Shader" : "unpredictableUnderScoreNaming", "version" : 1 }

# this is not correct / undesired !
# The Asset is miss interpreted to my_test_asset_blue_some
# the Shader is miss interpreted to unpredictableUnderScoreNaming
# my_template.validate("my_test_asset_blue_some_unpredictableUnderScoreNaming_v001") returns True but should be False

(*Note: a work around could include adding more template definitions, composed of other templates. Or a even an extra check for the “Asset” field to be a valid asset in the database. I want to avoid these work-arounds and if possible utilise template definition key options shotgun_entity_type and shotgun_field_name or a more convenient solution when working with fields that allow ‘_’ in general. )

The key options are documented here (The key section):


the templates.yml would contain:

keys:
    version:
        type: int
        format_spec: "03"
    Shader:
        type: str
    Asset:
        type: str
        shotgun_entity_type: Asset # (?) what does this do ?
        shotgun_field_name: code
strings:
    my_template:
        definition: {Asset}_{shader}_v{version}

1.) How do I use the template key options shotgun_entity_type and
shotgun_field_name correctly ?
Could you give a use case example ?

2.) What could fields that allow for underscores be handled neatly inside inside the templates ?

(*Note: there is also the option to specify a regular expression to valuate filter_by:'^[0-9]{4}_[a-z]{3}$', but in the case of handling/ variable multiple underscores for the Asset field, the regex will always output multiple solutions, which in return will make the template fail the validation )


Any help or advice is greatly appreciated.

Kind Regards

Stu

2 Likes
2

If you don’t expect underscores in the shader name, what happens when you set the type of the shader to alphanumeric ?

I’ve found that’s helped resolve unknown underscored fields in templates before.

1 Like
3

Hey Stu! Welcome to the forums! :blush:

If I’m understanding your question right, hopefully this is some information that will be helpful:

  • As you mentioned, you get ambiguous results when you are filtering by underscore and also using underscore as a delimiter between template keys. You could address this a couple different ways:
    • Modify your templates so that they use something other than underscore as a delimiter, and modify the regex filters so that they do allow underscores
    • Keep things as-is: do not allow underscores in your template keys; leave underscore as the delimiter.
  • As for shotgun_entity_type and shotgun_field_name, understanding these keys requires a bit of background…

The path cache

Toolkit uses something called the path cache to store mappings between folders on disk and entities in Shotgun. (More details here) Path cache entries are generated at folder creation time, and they’re determined by the schema. For example, if you create a path like /mymovie/sq100/s001 (very simplified of course), you might be creating 3 path cache entries:

  • /mymovie -> Project mymovie
  • /mymovie/sq100 -> Sequence sq100
  • /mymovie/sq100/s001 -> Shot s001

When it comes time to write files to this path, using the templates, Toolkit is actually going to these path cache values, and not the actual names of the Project, Sequence, and Shot from their entities in Shotgun to determine the values of the {Project}, {Sequence}, and {Shot} template keys.

shotgun_entity_type and shotgun_field_name

So, if you wanted to reference some other Shotgun entity in your filename that isn’t in the folder structure, there’s no data to resolve that value. This is where shotgun_entity_type and shotgun_field_name come in. Here’s the descriptions of these fields from the File System Reference document:

  • shotgun_entity_type - When used in conjunction with the shotgun_field_name option, will cause contexts to query shotgun directly for values. This allows using values from fields not seen in the folder structure to be used in file names.
  • shotgun_field_name - Only used in conjunction with shotgun_entity_type .

So, if you want to use some data from Shotgun entities in your filenaming that isn’t already in the file’s path, then you’d use the above fields to explicitly specify which entity to pull from, and which field to use, specifically.

You can see an example use case in that same document, here – this example uses those two fields to add a current user key to the file name.

I’m not sure if that fully answers your question. Let me know if I can clarify anything for you!

1 Like
4

Hey,

thank you two for the quick prompt response :slight_smile:

Good point. I found that specifying alphanumeric filters for some keys in a smart way, can solve problem to some extend.

@tannaz
thanks for the explanation. So that would mean, apart from the template evaluation, one would have to do another query to the context (to check for the asset name) to get the shotgun_entity_type option working ?

Do you have an example of how that could look in code ?

import os
import sys
import sgtk

tk = sgtk.sgtk_from_entity('Project', 391)
template = sgtk.TemplatePath(
    'work/assets/{asset_type}/{Asset}/{Asset}_{Shader}_v{version}.png',
    {'asset_type': sgtk.StringKey('asset_type', 
                                    shotgun_entity_type='Asset',
                                    shotgun_field_name='sg_asset_type'),
     'Asset': sgtk.StringKey('Asset', 
                                    shotgun_entity_type='Asset',
                                    shotgun_field_name='code'),
     'Shader': sgtk.StringKey('Shader', filter_by='alphanumeric'),
     'version': sgtk.IntegerKey('version', format_spec='03')},
    root_path=tk.roots['primary'])

path = (R'\\root\projects\newyork\work\assets\Character\test_asset\test_asset_my_shader_v003.png')

context = tk.context_from_path(path)

fields = context.as_template_fields(template)

fields.update(template.get_fields(path, skip_keys=fields.keys()))

Cheers
Stu

1 Like