Pip install sgtk breaks ToolkitManager

[Fabian]

Hello everyone!

I encountered an issue when I install tk-core via pip, that I cannot bootstrap an SGTK instance via the toolkit manager.
I am running this code:

import tank

sa = tank.authentication.ShotgunAuthenticator()

user = sa.create_script_user(api_script='SOME-USER',
                             api_key='SOME-KEY',
                             host='https://SOME-SITE.shotgunstudio.com')

mgr = tank.bootstrap.ToolkitManager(sg_user=user)
mgr.pipeline_configuration = "Primary"
mgr.do_shotgun_config_lookup = True
mgr.plugin_id = "basic.*"

engine = mgr.bootstrap_engine("tk-shell", entity={'type': 'Project', 'id': 123})

and get this error:

File "[...]\test_sgtk.py", line 31, in setUpClass
engine = mgr.bootstrap_engine("tk-shell", entity={'type': 'Project', 'id': 123})
File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 490, in bootstrap_engine
tk = self._bootstrap_sgtk(engine_name, entity)
File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 1098, in _bootstrap_sgtk
config = self._get_updated_configuration(entity, progress_callback)
File "[...]\lib\site-packages\tank\bootstrap\manager.py", line 1055, in _get_updated_configuration
config.update_configuration()
File "[...]\lib\site-packages\tank\bootstrap\cached_configuration.py", line 356, in update_configuration
raise TankBootstrapError(tank.bootstrap.errors.TankBootstrapError: Configuration could not be installed: Cannot execute hook '[...]\lib\hooks\bootstrap.py' - this file does not exist on disk!.

It’d be great if that would work so, because then I could run my tests in a virtual environment. Does anybody know how to fix this?

Cheers,
Fabian

[tannaz]

Hi Fabian –

I just looked this up, found a support ticket with the exact issue… and realized that you were the one that had submitted it. Sadly, not a lot has changed since you submitted that ticket nearly a year ago.

There are quite a few client requests attached to the internal ticket, so I’ll bring it up again with the team and see if we can shift it higher in the backlog.

Having said that, I can offer a bit more info, and a potential workaround to get it to work with newer versions of tk-core:

First off, some details on why it’s breaking: “The ToolkitManager has a dependency on the hooks folder, which is not included when pip installing tk-core, so bootstrapping is now broken when using a pip installed core.”

And a workaround that one client suggested – I can’t vouch that this will work for sure, but it might be worth trying:

# Define some required packages to be included.
DEPENDENT_PACKAGES = [
"shotgun_api3==3.0.40",
"sgtk==0.18.152"
]
# The syntax seen below for the shotgun_api3 package is expected,
# however, if we try to follow the same pattern for tk-core we get
# an sgtk package installed with the version "dev" rather than
# the correct 0.18.152. This is likely because of the get_version
# method found in tk-core's setup.py file. As a result, in order
# to download and install the correctly versioned package, we need to
# use the git+https syntax instead for that package.
DEPENDENT_LINKS = [
"https://github.com/shotgunsoftware/python-api/tarball/v3.0.40#egg=shotgun_api3-3.0.40",
"git+https://github.com/shotgunsoftware/tk-core.git@v0.18.152#egg=sgtk-0.18.152",
]

Let us know if it works!

Hope that helps! I’ve linked this post to the internal bug ticket, so we’ll be notified when it’s resolved, and will notify you here in turn (in addition to on that old ticket).

[Fabian]

Hi Tannaz!

Thanks for bringing the issue up again with the team!
I tested the client suggestion but it has the same result for me.
I think what causes the issue is that you are not including the hooks folder in the package data of the setup.py (and probably some other code adjustments). Hope that helps a little and crossed fingers that this can be fixed quickly.

[tannaz]

Yup, that’s it. These are the details from the internal ticket:

The ToolkitManager has a dependency on the hooks folder, which is not included when pip installing tk-core, so bootstrapping is now broken when using a pip installed core.
We should write a test that pip installs Toolkit and bootstraps into a config so we don’t break this in the future
One way to tackle this bug would be to
a) Move the hook base class inside the bootstrap framework (let’s say sgtk.bootstrap.BootstrapHook)
b) have the discovery logic use the hook from the bootstrap module

(Although, no guarantees that the suggested solution here would definitely be the way engineering decides to address the issue.)

Anyway, we’ll keep you posted as there are updates.

[Dan_Bradham]

Has there been any movement on this issue? I noticed that this is still a problem when I pip install tk-core.

[Jaziel]

After of rplicate the code i gt the same error .

So i performed an “archaic” solution, copy the hooks folder from git repo to the path of my python 2.7 (C://Python27/lib) and i get another error:

TankDescriptorIOError: Failed to download into path …\AppData\Roaming\Shotgun\bundle_cache\tmp\8c7ccb3d0c0b48ee9ce23b9bc1916d44: HTTP Error 404: Not Found

Surely this is an error on my project configuration, either way think this is not a real solution but maybe can help.

[Degraund]

My solution to the problem
ERR_PC - PC on which the error occurred
DONOR_PC - PC with SG Desktop of required configuration installed

1. Copy ~/.shotgun/bundle_cache/app_store from DONER_PC to ~/.shotgun/bundle_cache/app_store on ERR_PC.
2. Create a link ~/…/hooks to ~/.shotgun/bundle_cache/app_store/tk-core/vX.X.X/hooks/
3. Download https://github.com/shotgunsoftware/tk-shell.git to ~/.shotgun/bundle_cache/app_store/tk-shell/vX.X.X/