How to troubleshoot folder creation errors

Pipeline Integrations
, ,
1

[I’m gonna talk to myself here, as this issue comes up every once in a while, and I want to make sure it’s documented here :slight_smile: ]

This post addresses a couple errors that occur at folder creation time.

  • Could not resolve row id for path!
  • Database concurrency problems: The path <PATH> is already associated with Shotgun entity <ENTITY>.

Error 1: Could not resolve row id for path!

A Toolkit user is getting the error “Could not resolve row id for path!” when they create folders.

Oddly, this is creating FileSystemLocation entities, but sometimes resulting in duplicates, which can cause a whole host of problems.

The full error looks something like:

Creating folders, stand by...

ERROR: Could not resolve row id for path! Please contact support! trying to 
resolve path '\\server\nas_production\CLICK\00_CG\scenes\Animation\01\001'. 
Source data set: [{'path_cache_row_id': 8711, 'path': 
'\\\\server\\NAS_Production\\CLICK\\00_CG\\scenes\\Animation\\01\\001', 
'metadata': {'type': 'shotgun_entity', 'name': 'sg_scenenum', 'filters': 
[{'path': 'sg_sequence', 'values': ['$sequence'], 'relation': 'is'}], 
'entity_type': 'Shot'}, 'primary': True, 'entity': {'type': 'Shot', 'id': 
1571, 'name': '001_01_001'}}, {'path_cache_row_id': 8712, 'path': 
'\\\\server\\NAS_Production\\CLICK\\00_CG\\scenes\\Animation\\01\\001\\Fx', 
'metadata': {'type': 'shotgun_step', 'name': 'short_name'}, 'primary': True, 
'entity': {'type': 'Step', 'id': 6, 'name': 'FX'}}, {'path_cache_row_id': 
8713, 'path': 
'\\\\server\\NAS_Production\\CLICK\\00_CG\\scenes\\Animation\\01\\001\\Comp',

(actually it can go on much longer than that…)

Error 2: Database concurrency problems

Similarly, this error can also occur when a Toolkit user tries to create folders. Here’s the full error:

ERROR: Database concurrency problems: The path
'Z:\projects\SpaceRocks\shots\ABC_0059' is already associated with
Shotgun entity {'type': 'Shot', 'id': 1809, 'name': 'ABC_0059'}. Please re-run
folder creation to try again.

What do these errors mean and how do I fix them?

3 Likes
2

What’s causing the errors?

Error 1 points to a mismatch between the storage roots as specified in Shotgun (Site Prefs -> File Management) and config/core/roots.yml in the Pipeline Configuration.

It often comes up because of a case mismatch in studios running Windows. Their paths are case-insensitive, but our configs are case-sensitive. A difference as simple as E:\Projects vs E:\projects can cause this error.

Error 2 seems related, though I’m not exactly sure why it happens. It happens when you’re in a state where you’re trying to create a FilesystemLocation entity for a folder that already has one.

What’s happening behind the scenes?

In the case of Error 1, I believe what’s happening is this: The code creates the FilesystemLocation entity in Shotgun for the path it just created, using Shotgun’s storage roots to determine the root for the path. Then, it goes to create the same entry in the local cache, and has to determine where to put it in the database. For the local cache, it uses roots.yml to determine the root for the path, and because of the case mismatch, the path it generates doesn’t match what was just entered in Shotgun. At this point, it throws the error.

It’s especially bad because it doesn’t error cleanly: the folders are created, the FilesystemLocation entries are created, they are not synced in the local path cache, nor will they ever be able to be synced because of the storage root mismatch.

How to fix

  • For Error 1 only, First, make sure the storage root paths in the Site Prefs match the paths in config/core/roots.yml . Fixing the mismatch should make the error go away in subsequent folder creation calls.
  • [The rest of the steps apply to both Error 1 and Error 2.] Clear the bad FilesystemLocation entities. If you can narrow down to a set of errant FilesystemLocation entities, just remove those. In a lot of cases though, all of the paths for a project are compromised, so they all need to go.
    • How to clear the FilesystemLocation entities: ideally you can run tank unregister_folders . To clear all of them, run tank unregister_folders --all . (For all the options for tank unregister_folders , just run it with no arguments and it will output usage notes.)
    • However, because the db is already in a wonky state, this may not work, or may only partially work. Once you’ve run the command, go back to FilesystemLocations in Shotgun, and confirm that what you expected to be deleted is actually gone. If not, select the bad entities, and move them to trash manually.
  • At this point, the FilesystemLocations in Shotgun are clean, but artists’ local caches may not reflect your changes. The last step is to actually sync the local cache on each user’s machine. To do this, they should run tank synchronize_folders --full .

Once all those steps are taken, the path cache should be in a good state, and that errors shouldn’t appear anymore.

Related Links

7 Likes
4

Edit: I’ve modified the original post to include the below error.

I’ll note that the following error can also be a symptom of the same issue:

ERROR: Database concurrency problems: The path
'Z:\projects\SpaceRocks\shots\ABC_0059' is already associated with
Shotgun entity {'type': 'Shot', 'id': 1809, 'name': 'ABC_0059'}. Please re-run
folder creation to try again.

I’m not sure how the db ends up in this state, but above “How To Fix” steps, save for the first one about roots.yml, should also apply to this error.

3 Likes
9

So I have a similar error. Creating a folder with an asset of the same first letter as an existing one seems to think its already registered.
For the schema in question we are placing assets in a letter subdirectory so that it looks like:
assets/Prop/C/Crab
But then the registered file for this ends up being assets/Prop/C
So creating another C asset causes this error.

Does something have to be registered with the schema so that it recognizes the proper Asset root and not the parent directory?

This is the yaml for the initial directory

type: "shotgun_entity"

name: "{code:^([A-Z])[a-z]}"

entity_type: "Asset"

filters:
    - { "path": "project", "relation": "is", "values": [ "$primary" ] }
    - { "path": "sg_asset_type", "relation": "is", "values": [ "$asset_type"] }

And this is the yaml for the asset directory

type: "shotgun_entity"

name: "code"

entity_type: "Asset"

filters:
    - { "path": "project", "relation": "is", "values": [ "$primary" ] }
    - { "path": "sg_asset_type", "relation": "is", "values": [ "$asset_type"] }

Anything here I might be missing or configuring incorrectly?

Thanks!

10

Scratch that! I fixed it.

I consolidated the schema into just the asset directory’s yaml file and it works now.

type: "shotgun_entity"

name: "{code:^([A-Z])[a-z]}/{code}"

entity_type: "Asset"

filters:
    - { "path": "project", "relation": "is", "values": [ "$primary" ] }
    - { "path": "sg_asset_type", "relation": "is", "values": [ "$asset_type"] }

Sorry to re-open this topic!

11

We’ve been having this issue and unfortunately none of the above solutions are solving the problem.

So’m getting this error:


ERROR: Database concurrency problems: The path
'M:\project\PATH\TO\ASSET\Step' is already associated with SG entity
{'type': 'Step', 'id': 340, 'name': 'Model'}. Please re-run folder creation to
try again.

I followed the steps above:

So at this point all paths are unregistered and there is not FilesystemLocations in ShotGrid.

Then I ran tank synchronize_folders --full

No errors so far. But as soon as I try run the folders command or launch Maya, I get the same error.

Is there anything I’m missing?
Is there a to wipe down the entire paths?
What can I do at this point?

1 Like
12

As troubleshooting step, try renaking your ShotGrid cache folder so SG desktop has to cacheit all again.

13

Did that as well, no luck

14

Hi Ali!

You might want to check that there isn’t a discrepancy in your storage roots, per this part:

Particularly if you are on Windows, the OS is case-insensitive while our systems are case-sensitive, so if there’s a case discrepancy between roots.yml, site preferences, and/or the folder name on disk, that might be the culprit.

Good luck!

15

Much appreciate it Tannaz.
Yeah I already tried that, I tried everything that was mentioned on this forum.

I’m just frustrated by the fact that if there is no FilesystemLocations entity, and there is not path_cache.db and already ran the unregister_folders --all, where does this come from Database concurrency problems? where is this database that tank is reading coming from?

16

Good question – I’m not sure when in the process the sync takes place when creating folders.

You could try not taking doing the sync step? Clear Filesystem Locations, delete path_cache.db manually, then create folders and see if that works? This is a weird one for sure.

17

same result, unfortunately.

  • Closed SG Desktop app
  • Deleted the entire AppData\Roaming\Shotgun directory
  • Made sure the windows_path in the root.yml is the same exact as the what is set in the Site Preference settings
  • Made sure there is no FilesystmeLocation entity
  • Launched SG Desktop app
  • Ran the tank unregister_folders --all
  • Ran the `tank Task folders
  • Got the same error:
ERROR: Database concurrency problems: The path
'M:\project\PATH\TO\ASSET\Step' is already associated with SG entity
{'type': 'Step', 'id': 340, 'name': 'Model'}. Please re-run folder creation to
try again.
18

I would get in contact with support, maybe some stuck cache somewhere.

19

Yeah submitted a ticket to them but no response yet, hopefully soon :crossed_fingers:
Once we figure it out, It’ll let you know in case someone else stuck the way we are :sob:

1 Like
20

So finally we track down the issue, which was caused by have 2 different Model pipeline step on our assets but with the same short name.

Once we cleaned up the steps and made sure they have different short names, the folder generation started working.

I hope it won’t happen to anyone else because it was pretty troublesome to track this down but now we know.

Thanks to the ShotGrid engineering and support team for looking into this.

2 Likes