Shotgun Desktop: force refresh when config is out-of-date

Pipeline Integrations
1

We have some users (internal artist and external vendors) that are inclined to keep SG desktop up and running for days/weeks on end without a refresh. The frequency of configuration updates is on the order of at least once a week, so there’s a risk of users running stale tools (backed up by anecdotal evidence).

Nothwithstanding that a strong studio policy (i.e. forced reboots) would go a long way here, are there any toolkit mechanisms for forcing an automatic refresh when the loaded configuration is out-of-date?

CC’ing @vincent.wang

3 Likes
2

Hey Nico! I’m fairly certain that any time you bootstrap (which happens every time you launch a DCC with an engine from Desktop), that it’ll check for updates to the project’s config if it’s descriptor-based, and update the cache. Is that what you’re after?

On the other hand, I don’t think the site config gets updated until you relaunch Desktop.

3 Likes
3

Unfortunately, the case I’m describing is for non-DCC apps that are configured for the desktop engine, so there’s no secondary bootstrap happening. Best I can think up with is some kind of global pre-app launch hook, assuming this is an event that core exposes via a hook?

4

Hi Niko,

Best I can think up with is some kind of global pre-app launch hook, assuming this is an event that core exposes via a hook?

This is the exact way we ended up handling this issue. We do a checkup in before_app_launch.py hook on tk-multi-launchapp which is called before any configured application is launched via Shotgun Desktop.

At the moment, we do not force the update (although it has been considered) but instead we show a big-bad-notification window and ask the user to kindly upgrade his/her configuration before proceeding. Definitely not the ideal mechanism - particularly for the artists who tend to ignore warnings - but much better than no measures at all.

One thing I’d like to warn you is to be careful when comparing versions. Depending on how you distribute your configs, it may not be the issue, but I’ve found that descriptor.find_latest_version() takes a lot of time, and in any case, it will return the actual latest version it found for the config not the latest “defined” version. Since we rarely allow project to use “latest version” and prefer to specify them, this is really not useful all that much. Instead, I’ve used the API to get the descriptor URI as defined on shotgun site for current project and config name, and than regex-parsed the version string which is than compared to the current descriptor version. I’ve tried other things but this proved to be fastest which was important since adding to already long startup times is a major annoyance for everyone.

Cheers,
Aleksandar

3 Likes
6

Thanks Aleksandar,

But in the case of tk-multi-launchapp, aren’t you bootstrapping anyway by virtue of starting a new engine? In our case we’re dealing with toolkit apps configured to run in the desktop engine directly (e.g. ‘tk-multi-publish’) so we don’t have the benefit of a second bootstrap nor is tk-multi-launchapp an entry point, which is why I was fishing for core hook of some sort that would catch all these cases.

Thanks again
Nico

2 Likes
7

Hi guys – Aleksandar, thanks so much for sharing your solution.

As Nico mentioned, it gets us part of the way there.

Nico, I talked about this with a couple people from the team earlier, and there’s not really a hook currently that would run at app run time. What you could do is add the logic that Aleksandar is suggesting to your apps themselves. Not the most elegant, but it would get you there.

Also, it seems like it’d be useful to have a hook that does run before an app is run – this comes up for the standalone Publisher as well as custom apps. I’ve shared that feature request with the product team.

1 Like
9

Yup, putting the logic directly in the app was actually our initial impulse. It led me to thinking that we might want to externalize it and write a catch-all, but we can definitely run with this implementation in the near-term. A new pre-app-lauch hook would be awesome (I’d rank it a “would-be-nice” over a “must-have”).

Cheers

1 Like
10

Agreed! I linked you to the feature request (it’s internal, but we do associate them with specific clients), so you’ll be notified if there’s any movement there.

11

Ah, sorry for not reading your post more carefully, Nico. My apologies.

12

Your answer still contained useful info though, thanks for sharing!

One detail I did want to mention, which doesn’t really apply to Nico’s question, but may still be important to know, by default, launching software won’t actually check and cache the latest versions.
I just ran some tests, and if you are using the default2 config, launch Maya from SG Desktop, and then run this code in Maya, you can see that it is using the cached config from the desktop engine (note p89c331.basic.desktop in the sgtk module path).

import sgtk
print sgtk
>> <module 'tank' from '/Users/philips1/Library/Caches/Shotgun/mysite/p89c331.basic.desktop/cfg/install/core/python/tank/__init__.pyc'>

engine = sgtk.platform.current_engine()
engine.sgtk.configuration_descriptor.get_path()
>> '/Users/philips1/Library/Caches/Shotgun/bundle_cache/sg/philscad/v3942'

However, if you set launch_builtin_plugins: [basic] under your Maya project engine settings, then this will mean that it will perform a full re-cache of the latest config when it bootstraps in Maya. This won’t affect the current SG Desktop session, but it will mean that Maya is using the latest config.

Now see the same code run from Maya after using this setting

import sgtk
print sgtk
>> <module 'tank' from '/Users/philips1/Library/Caches/Shotgun/mysite/p89c331.basic.maya/cfg/install/core/python/tank/__init__.py'>

engine = sgtk.platform.current_engine()
engine.sgtk.configuration_descriptor.get_path()
>> '/Users/philips1/Library/Caches/Shotgun/bundle_cache/sg/philscad/v3943'

You can see that the cached config is now specific to Maya. Also since I updated my uploaded config after choosing the project in SG Desktop but before launching Maya, you can see that the attachment id for the config is now v3943 rather than v3942 in the first test.

So if you set launch_builtin_plugins: [basic] on your engine project settings, then it should at least pull the latest config when you launch software, though it will only apply to that software and not SG Desktop. That setting can be applied to most of our engines but not all of them, check the engine’s info.yml to see if it supports that.

Small caveat, if you do set that setting, on the config, you will still need the users to jump out of the project and back in, in SG Desktop once, just so the setting is loaded from an SG Desktop, then any future config updates will be cached when launching software, even without re-entering the project in Desktop.

13

no worries Aleksandar. Always informative to hear feedback and find out how everyone’s solving similar problems.

2 Likes
14

This is handy info!

Is this heavy on the network? I would imagine the caching is smart and only changes files that are actually changed?

As a side note, It would be great if we could define some sort of Env Variable telling desktop how many times in a day/week/month it should automatically check for updates and could trigger that the moment it lapses and an application is launched.

Another side note, if you run Windows and have deployment software like pdq deploy/inventory, if you run deadline/your render manager in user space or have access to psexec you could just command the machines to terminate all shotgun.exe processes the moment you want to force your update :wink:

Do it before people start their day and voila, they will have to restart it :wink:

1 Like
15

It should only download something if the descriptors are pointing to a version that is not currently in the bundle cache.

1 Like
16

You mean, an app version?

It wouldn’t update scripts like hooks for example?

17

For any bundle, i.e. an app, engine, framework, core, or a config.
Hooks wouldn’t be cached individually, but usually, they are stored in the config, so they will be cached with that.
If the specific version of the required config is not in the bundle cache, then it will be cached.

1 Like
18

The Law of Regrettable Automation: Immediately after modifying software to self-upgrade is the moment when the latest version of that software becomes unstable.

4 Likes
19

I was just re-reading this and thought to mention, doesn’t the pipeline_configuration_init core hook run every time you start an application?

If you add some logger info in there you should see it turn up about everywhere.

Also, any update on if there are better ways to force updates?

1 Like
21

Hi Ricardo,

Happy to be proven wrong here, but I don’t think pipeline_configuration_init would be invoked unless a new engine is initialized, which shouldn’t be the case for any non-DCC apps like sgtk-multi-publish2.

1 Like
22

Put a log statement in there and see for yourself :wink:

If I launch the standalone publisher it definitely gets run!

1 Like
24

If it worked as you described Ricardo, it would certainly open up the possibility of a force refresh we’re looking for, but unfortunately I just tested with a log debug statement and in our case the hook only gets invoked once when the shotgun desktop engine gets initialized. All subsequent invocations of apps that are configured to run within the desktop engine and don’t require a new pipeline configuration instantiation, so the hook doesn’t get run. If this isn’t the case for you maybe it’s a function of the way you’re launching standalone publisher? Quite possibly you’re running it in another engine?