Updating Pipeline Configuration

Hey there,
I have some questions about sharing Pipeline Configuration and dev sandbox.

From the dev help page and topics How do I find my pipeline configuration? and Shared project configs I understand that I can have a default Studio configuration.

It is also possible to use different configurations per project, depending on the User Restriction field on the Pipeline Conf webpage.
I need some more informations:


:diamonds: The help is pretty vague about what folder path actually must be used in the Pipeline Conf page:
• Windows Path shows the sgtk_projectName path
• Descriptor seem to need the sgtk_projectName/config path

Is that correct? It seems logical (it would basically tell the project to use this project specific tank, etc created during the Advanced Install, but use those config files).
Can anyone confirm this?


:diamonds: What does using a studio configuration mean exactly?
Is it possible to have a /config folder used by all my projects, with all projects having only some customized hooks in their own /config folders?

(eg: on projectName, if I only need to modify userSetup, then hooks/tk-maya/userSetup.py would be the only file in the project config folder while other hooks would be loaded in the studio config folder?)

Or is it just like the sgtk default config2 (the project will use 100% of this one studio config and that’s all) ?
If I just want a very small change in a project, do I have to completly fork the whole project config?


:diamonds: I initially created a Sandbox project, and I want to duplicate it’s configuration to use it as my Studio config.
I’d like to use git as version control, so that I can update my Studio config from my Sandbox config easily, so basically I just push my sgtk_sandbox/config folder to a git repo and use a branch in the descriptor field of my project to use it as my config.

Logically, once this is done, can I completly delete the original /config created from the advanced setup wizard? This should logicaly be useless now?


:diamonds: I have a hard time with the setup of the descriptor when using a git url: I think I have the wrong “user@host”: for gitlab, is it “git@gitlab” ?

This template path does not work and crash SG Desktop when trying to load a project:
sgtk:descriptor:git_branch?path=git@gitlab:companyGitName/folder/repoName&branch=branchName


:diamonds: Is it ok to use tk.pipeline_configuration.get_all_platform_data_roots() when the configuration is on git ?


Thanks!

3 Likes

If you’re using a centralized config setup via the SG Desktop advanced setup wizard then the PipelineConfiguration entity will have the Windows, Mac, and Linux path fields filled in. These paths point to the root of the configuration which should look like this:
sharing_caring_core

If you are using the PipelineConfiguration entity’s descriptor field instead of the Windows, Mac, and Linux path fields, and the descriptor should point to a source that contains only the uninstantiated config. Such as this: GitHub - shotgunsoftware/tk-config-default2: The second generation default configuration for ShotGrid Toolkit..

So you are correct in a way, because the “config” folder in a centralized setup is the config that you need to point to in a descriptor based setup. However, when I say uninstantiated, I mean that the config folder should contain the following files:

  • ./core/pipeline_configuration.yml
  • ./core/interpreter_{OS name}.cfg
  • ./core/install_location.yml
  • ./core/shotgun.yml

And the roots.yml usually doesn’t have the actual paths filled in.

So in short what you were saying is correct. Though a PipelineConfiguration setup with a descriptor only needs the config folder, and not the install, cache, tank file.

Studio config is not really a very well defined term, unfortunately. I’m not quite sure on the context around where you saw it mentioned, but it can sometimes be refer to a shared core. A shared core being where you have run tank share_core /some_path/studio and set up a “studio” core that all other configs would be attached to. A shared core means that you share the tk-core API, the bundle cache (ie all downloaded apps, engines, and frameworks), and some small specific elements of the config. However, the project specific configs attached to this shared core, still maintain their own separate configuration file such as schema, template environment YAML files.
There isn’t a concept of overriding settings at a project specific level from the shared core. Though overriding hooks without changing the config is something we have thought about before, I’ll forward this on to our feature request portal.
I guess a Studio config could also refer to maybe a site config, which is a non project-specific config that will be used by default by all projects unless they have their own config, which is maybe what you’re referring to as a studio config?

I think what your mentioning there makes sense. Might worth taking a look at this:

I’ve not used this syntax before either actually, so I’m not totally sure, but the path variable bit (git@gitlab:companyGitName/folder/repoName) is purely git syntax and nothing specific to Toolkit.

I’m not familiar with that method, and it doesn’t appear to be in our API documentation, however, I think the question would be more would it work with a distributed config vs a classic installed config. I tested it with a project using a dev path descriptor and that work fine, so I think the answer would be yes. The config would have to be bootstrapped before you could make such a call, and once it’s bootstrapped it’s cached locally and no longer in git.

Summary

It sounds to me like you are considering switching from a classic installed config, also sometimes referred to as a centralized config, to a distributed config setup. And you are thinking of having a site config so all projects work off the same configuration by default?

1 Like

Hi Philip,
Thanks for your answer, it’s more clear now.
I will stay on a centralized setup, use a default studio config to setup my project and use git push/pull to update different project configs. This seems like the safest way for now and I should be able ti user .gitignore to avoid copying files in /core.


A global studio config that could be overrided per project by simple hooks would be an awesome feature!


About the git “scp-like” ssh path, i’d have to use the actual username:
sgtk:descriptor:git_branch?path=username@gitlab:companyGitName/folder/repoName&branch=branchName
SG Desktop will then launch a cmd prompt to type the user password (but it is impossible to input anything, as stated in the doc privates repo are not supported.)


Do you know how I can query the config path setuped in the descriptor? Methods like would give a path to the project /install folder, not to the config:
sgtk.get_sgtk_module_path()

This should work, but the doc don’t explain what the arguments to instantiate the class are:
sgtk.descriptor.ConfigDescriptor.get_path

Thanks!

1 Like

The class instantiation is described here but usually you wouldn’t instantiate the config descriptor yourself. You can get the sgtk instance from the engine usually, and from there you can get the config descriptor:
https://developer.shotgunsoftware.com/tk-core/core.html#sgtk.Sgtk.configuration_descriptor

So you can do something like this:

tk.pipeline_configuration.get_all_platform_data_roots() 
# >>  {'primary': {'win32': None, 'linux2': None, 'darwin': '/sg_toolkit/projects/my_project_config'}}
2 Likes

Thanks a lot!

1 Like

The correct function is:
tk.configuration_descriptor.get_config_folder()

tk.pipeline_configuration.get_all_platform_data_roots() will return the local temporary repo of the config

2 Likes