Adoption challenges

Hi Shotgun fam! I spend a lot of time thinking about how we can make it easier for people to get started in Shotgun, so I thought I’d start a thread for people to share the biggest challenges they faced when getting Shotgun setup for your team/department/studio. The more you share, the more we can smooth out those speed bumps and make it way easier the next time you or someone else needs to get things humming quickly in Shotgun.

As an aside, something to keep in mind is that a lot of what makes it challenging to get started in Shotgun is just taking time to document and polish your workflow and communication strategy. Sometimes a studio has all of this figured out and just needs to drop it into the Shotgun schema, but in other cases, this is the first time you have made a concerted effort to define how things work in a more formal and repeatable way, that is visible and transparent to everyone. This is tricky stuff, no matter what software you are using. But well worth the effort!

Thanks in advance for your input and insights! Will also make sure and highlight in this forum when we have new stuff rolling out to make everyone’s lives easier on the adoption front.

10 Likes

There’s three types of challenges - for the users, for development, and for design. For design - you don’t always know exactly where you’re headed. There is so little details about pipelines out there that it’s very difficult to get some advance knowledge before messing things up. For development - since there’s multiple paths to getting to the same goal, it’s difficult to figure out what are the pros and cons of a plan - weighted against a big plan rather than step by step, which the docs cover alright. For the users - adopting a new and often continually changing platform, which we do a lot of internal talk and training to get across :slight_smile:

7 Likes

Raise some voice for my clients. Most people will finally be convinced that Shotgun is a robust tool, but it takes them quite a while to get that feeling. As Hristo_Velev mentioned in details that a clearer product tour (would be nicer if it’s a role-base product tour) is of necessity for tools as robust and comprehensive as Shotgun.

3 Likes

(apologies in advance if this feedback is too low level, but here goes…)

I’ve been in the process of setting up a Shotgun Toolkit pipeline in a studio with an existing pipeline (which already uses Shotgun for tracking). We’re not fully up and running yet but I can share a few things that have come up so far, both from users and myself doing the dev/setup. I’ll continue to add to this thread when I remember more things too! :slight_smile:

Development docs
Module documentation is generally quite good but it would be great to have more of an overview that describes how all the parts fit together, from a technical perspective, but at a higher level than just the python modules. It has taken me a while to really ‘get’ how toolkit works internally, but things are coming together more easily the more I understand the big picture.

One thing that took a while to understand is knowing what code is actually being run and when. Now I understand that the apps are downloaded into ~/.shotgun/bundle_cache and that the core sgtk modules are in ~/.shotgun/p{project_id}.plugin_id.* it feels less like spooky action at a distance.

Something that doesn’t help here is outdated documentation floating around the internet (including old posts on shotgun-dev etc). I always have to approach everything I read with a bit of skepticism since it seems many methods and recommended ways of doing things have changed over the years.

I’m using an uploaded configuration and much of the info I’ve found online barely touches on this, if at all. I assume it’s because this is new but it makes things difficult. I don’t know if this is specific to my site here, but the plugin id and uploaded config columns don’t even show up by default in my Pipeline Configurations tab. The SG Desktop Advanced Project Setup doesn’t clearly refer to it either.

Render farm usage
It’s commonplace to want to shove as much processing as possible on the render farm, not users’ machines, and to use the farm to automate procedural pipeline processes. It seems that Toolkit hasn’t really been designed that much with a farm in mind. I can understand this since it needs to be workable for all types of studios, including those without farms, but it would be great if it supported farm workflows better out of the box.

For example the out of box workflow of needing to convert Nuke write nodes or Houdini ROPs to the stock versions before sending to the farm is really nasty for the user and adds manual processes that can be error prone, to what should be easily automated. On the flip side, the SG Write Nuke node hides useful parameters that are on the default write node, and same with the Houdini SG Mantra node (when it’s a hardcoded HDA from Shotgun it won’t update with new features in new Houdini versions).

Most (larger) places that I’ve worked at have usually had things set up so that you can expect a farm blade to have access to the exact same software environment and let you do the same things as on a local workstation. If this is not the case it can make things a bit of a nightmare to debug. In our case here it’s taken me quite a bit of pain and effort to set up a bootstrap system on the farm so we wouldn’t have to do the SG node conversions and to allow us to use the SG context to run procedural processes on the farm. I’ve also ended up duplicating/extracting some of the python code from the SG mantra ROP and just using that as default parameter expressions/spare parameters on the default Houdini Mantra ROP, to get the best of both worlds (access to shotgun context + not stuck with SG nodes). I’m a bit concerned about what might happen when things get going though since I’m told this could hammer shotgun too much and get slow. For now this is the best I’ve got out of some imperfect options.

The other main thing we’re missing is being able automatically create Versions from Nuke or Houdini renders with associated Quicktimes automatically after they are done on the farm. Currently it looks like this has to be somehow done manually with the SG QuickReview, or I’ll have to try and script something with Deadline Draft etc. This is a vital part of our workflow and it would be nice if it worked more seamlessly out of the box.

8 Likes

Great feedback, fits with some of our experience too. On the last part - we have built quite an extensive Draft system over time that is currently relying on JSON config/defaults and shotgun to process the quicktimes in the right way, depending on the context and user preferences. We’ll open source this at some point. It’s by no means trivial, and it’s a glaring gap in the out of the box tools. Another way to set it up is using Houdini’s standalone PDG, we haven’t delved into that much yet though, since our Draft system is working well.

4 Likes

I agree as well. It seems like all of the documentation and training come from a standpoint where it assumes you have all the knowledge needed about pipeline and the terminology. We’re a shop trying to develop a pipeline and adopt Shotgun at the same time. I’m often confused on the best practices. It does take a while to have users understand the depth and power of this platform. In the meantime users are quickly dissuaded by the interface not looking snazzy like so many other modern tools. That’s a tough barrier to fight through.

In short. Better UI. And some kind of training track to get less pipeline savvy users up to speed. Especially if you don’t have Dev resources on hand.

11 Likes

Hey Matt! Thanks so much for taking the time to share all this feedback—it’s extremely helpful to hear it. I wanted to let you know that on the topic of dev docs, we know, we agree with you, and it’s something we’re actively working on. Things are disparate and hard to discover, reference docs leave you with a lot of questions as to how to actually put everything together. As I work on Toolkit support, it’s something I feel myself—we shouldn’t need to point to you the right docs for the problem you’re trying to solve; you should be able to easily find good, useful information on your own.

So, we do have a new Developer Documentation portal that we’ve been working on for the past months. We’ve made a lot of progress, and there’s a lot more coming. It’s a clean fresh start, but it’s on us to be mindful not to just shift the existing issues to a new location. I hear your feedback, especially these points:

  • questions about local caching and how all the pieces are organized: Yes. Very murky waters.
  • outdated documentation: Getting rid of old stuff is a big priority. Also, @johnny.duguid, this is something for us to keep in mind as we consider options for shotgun-dev archives
  • uploaded config treated like second-class citizen: This is really important. It’s a paradigm shift for us, but as uploaded configs become more and more prevalent, it behooves us to bring them to the forefront in our docs.

Thanks again for bringing up these important issues!

4 Likes

Hey Byron! Ditto everything in my reply to @matt_ce here! We hear you, and we want to mitigate this pain! I’d be interested in more details: if you can call out specific workflows or specific documents that left you wanting, I’d love to hear it. Thanks again for your feedback.

2 Likes

Hi all, thanks very much for all the feedback here so far, and thanks @tannaz for the updates on the developer docs!

Please keep the comments coming everyone, and as a reminder to all the less-technically-focused folks out there, we’d love to hear about general Shotgun onboarding and adoption challenges, as well as the pipeline-focused input. The goal is to make it easier for everyone to get started with Shotgun!

4 Likes

I’ve implemented Shotgun TK at a few companies and, consistently, the problem is getting programmers onboard.

The initial learning curve for SGTK is steep there’s a tendency for most programmers to say “its just too complicated” and “I can write something simpler that will do the job”. Often that winds up atop the SG Python API3 anyway, since that’s relatively contained and comprehensible (even though most immediately wrap it in an “object API” which often is not a good idea due to the major differences between relational data models and object models).

So given that Python API3 is acceptable why not the rest? In short, the requirement for engines. SG Engines are amazing, but require swallowing the whale whole… and giving up launch primacy in existing in-house made apps.

What would help is a way to incrementally add modules from SGTK to your existing code. E.g. “let’s try using the context module” or “let’s call the folder module from our existing code”.

Its still a bit steep given the need to manage the configuration, but the default’s gotten much better and cleaner there.

SGTK is, frankly, an amazing bit of software engineering, but if it were more of a toolshop and less of a walled garden that would help adoption.

8 Likes

Thanks for saying that. Is there a particular part that puts people off do you think?

4 Likes

One thing that would improve interaction a lot is to make the fields editable with the dropdowns when multiple entities are selected. As opposed to have to right
click and bring up another dialog. By default, you may consider adding some color context. I know that’s a very subjective thing, but when you show off Shotgun for the first time it can come across like a glorified spreadsheet. Most of my users here have come
to appreciate the depth and strength of the platform. But the initial sell can be tough, especially when you have people looking at modern tools like Monday.com, Asana, Trello, etc… I’m really enjoying Shotgun, and I’d like to see better adoption both internally
and externally. Anything to make it more intuitive is a boon. Thanks for listening!

6 Likes

Can I just check which of these interactions you’re referring to?

2 Likes

Oh, I meant for example: If you select multiple shots and want to change a field, it would be nice to then be able to click in the dropdown of one of them and
have it update for all selected entities(shots). Some other (competing) tools have this functionality. I assume it’s something with how the UI is created in the browser.

4 Likes

Makes sense. Thanks

2 Likes

Hey Ron, great to see you here!

This is something we’re thinking a lot about. There are ways to initialize the TK bootstrap when just double clicking the app icon, but it does require studio (and OS) specific setup. We’ve been trying to target more and more functionality that “just works”, although we’ve only been able to get a subset of what TK can do there.

What kind of setup are you imagining? Also, when you are looking to use engines without “swallowing the whole whale” :wink:, what kind of things are you looking to do… especially those that target those devs just looking to get started with sgtk without needing to know everything?

4 Likes

I work as a VFX editor client side. Been using SG to keep track of shots and metadata for almost 5 years now.

I feel that this side of Shotgun users are slowly let down as is not as exciting as the vendor side usage.

I feel that there should be a few training videos and documentation that would deal specifically with this side and also a more tailored TV default project.

It’s been 5 years and I’m still frustrated about the fact that I can’t change the framerate of a project within SG.

I would also love to see more integration with NLE’s. Like Media Composer plugins.

Also because I have 0 programing knowledge I’d be more than happy to pay for some plugin scripts that will make my life easier.

11 Likes

Simplicity has been our largest hurdle. Most people are coming from tracking methods that are more simplified/less interconnected. While Shotgun is great, it can seem very daunting to wrap your head around when you are first onboarding someone.

Things that I would love to see, in order to help with this:

  1. Alternate/customizable “My Tasks” UI. We have to make alternate My Tasks pages for almost everyone. None of our users like the default “My Tasks” page unfortunately. Shotgun Create is a step in the right direction but would be great to support preferences in the web interface.
  2. Drag and drop - it would help A LOT if there was a drag/drop functionality for artists’ tasks. Especially when moving people from JIRA/Trello/ and other forms of Kanban. The simplicity of that method has been very difficult to reproduce.
  3. Training curriculum - there’s a maze of videos and tutorials that you can seek out for yourself, but it would be great to have a couple curriculum tailored to Artists, Production, Supervisors. We have been compiling these ourselves of course but you never know if you’re missing the latest best practices.
6 Likes