Cannot import sgtk but it is in path

Hi, on the farm we perform some automation with toolkit. Rarely but consistently (say, 1% of cases) we get Importing sgtk failed: No module named sgtk.
For debugging we print out the python path immediately before this, and it clearly contains the path to toolkit projectpath/install/core/python, which is added at the start.

I am out of ideas how to further debug that. There does not seem to be a definite pattern to the failures (no specific machines, types of jobs).

1 Like

A few thoughts:

  • Is it possible that the farm machine is not able to see the path properly?
  • Maybe the drive isn’t mounted successfully?
  • Corrupt .pyc file or .py file?

Perhaps try adding to the script importing sgtk a line that in the event of an import failure, it will open the sgtk __init__.py and print the contents. I’ve seen instances when I was a TD where the filesystem showed the file as existing but had no contents but on another machine it was fine.

2 Likes

Thanks a lot, will try your suggestions.

Exaclty like suggested by @philip.scadding, I sometimes have a similar issue on some rendernodes:

A path can be in PYTHONPATH but not available locally.
To be sure all machines are using the same versions, libs are hosted on a server and mounted on nodes.

If your setup is a bit similar, I would check that first.

Thanks, yeah specifically toolkit libs and config are on the network (mounted drives in windows).
It seems it would require specific conditions to happen then, the drives would become unavailable at exactly the point of import sgtk.

Since this usually happens on Deadline - it mounts the drives on every job/task start.

Still have not tried out printing out one of the files :blush: was out for a while.

As per your suggestion I tried to open and print a file (__init__.py) from sgtk, and sure enough it is not there.
It is possible that Deadline has not mounted the network drive at this point, somehow, although it is supposed to mount them on each job start.
We do not have any evidence it is caused by our infrastructure at this point.

Alright just now that I said it, there was another case, where __init__.py was accessible (contents were printed out in the log) but the import still failed. Could it be that I have to sleep for a bit to wait for the drives to be available? That doesn’t make too much sense.

That is odd. I’m not really sure what to suggest there.

It would be weird and messy but after the reading of the file can you import it? If you were then potentially you could loop until the file had contents and then import. But that would be really not nice.

It is really baffling. At this point I think it should have something to do with how Deadline mounts the drives. “Event plugins” (where we get the error) are sort of second-class citizens, which run in a different context from Render plugins. Might mean that mounting may not have occurred, or was affected by another job.
Going to look around the Deadline forums.
Thanks a lot for your help!

1 Like

Wrote on the Deadline forum Event plugin and mounted drives - Deadline - AWS Thinkbox Discussion Forums if anyone is interested.

1 Like