Query All projects using REST Api

Pipeline Integrations
1

I’m trying to figure out how to query all active Projects and secondly query the Assets of a project, using the project name.

Below is a snippet of code demonstrating this exact feature in Python. However I would like to do this using the REST Api. I’ve attempted to look into the docs and create this query but have struggled to understand it. My goal is to use the rest api so i can place my queries in javascript.

tickets_getEntitiesProjects.py (2.2 KB)

Any help would be greatly appreciated. I’ve create a trial shotgun website to test this stuff one. Please be kind as I’m sharing my credentials for it. :slight_smile:

################################################################################
# Imports
################################################################################
import os, sys, pprint
import shotgun_api3

################################################################################
# Variables
################################################################################
SERVER_PATH = 'https://somesite.shotgunstudio.com/'
SCRIPT_NAME = 'aname'
SCRIPT_KEY = 'somekey'
sg = shotgun_api3.Shotgun(SERVER_PATH, script_name=SCRIPT_NAME, api_key=SCRIPT_KEY)

################################################################################
# Methods
################################################################################
def getProjectEntitiesByCategory(project='', category=''):
    '''
    Returns a list of entities in the shotgun project VFXP.
    '''
    filters = [
        ['sg_category','is', category],
        ['project.Project.name','is', project]
    ]
    fields = ['code', 'project']
    results = sg.find('Asset', filters, fields)
    results = sorted(results, key=lambda k: k['code'].lower()) 
    return results

def getProjects(category=''):
    '''
    Description:
        Returns a list of projects
    '''
    filters = [
        ['sg_status','is', 'active']
    ]
    fields = ['name', 'project']
    results = sg.find('Project', filters, fields)
    return results

################################################################################
# Testing
################################################################################
if __name__ == '__main__':
    results = getProjectEntitiesByCategory('VFXP', 'Application')
    pprint.pprint(results, indent=4)

    results = getProjects('Application')
    pprint.pprint(results, indent=4)
3 Likes
2

I uploaded the file for easier testing. Since the formatting is a little odd on here.

tickets_getEntitiesProjects.py (2.2 KB)

3

Hi @JokerMartini,

import json
import requests
import urllib

# Site URL
shotgun_url = 'https://somesite.shotgunstudio.com'

# Build API login url
login_url = urllib.parse.urljoin(shotgun_url, 'api/v1/auth/access_token')

# URL Encode for safety
url_credentials = urllib.parse.urlencode({
    'client_id': 'ascriptname',
    'client_secret': 'ascriptkey',
    'grant_type': 'client_credentials'
})

# Login using your script credentials
response = requests.post(login_url, data=url_credentials, headers={'Content-Type': 'application/x-www-form-urlencoded'})

if response.status_code == 200:
    # Get token type and access token from successful response
    _token_type = response.json()['token_type']
    _token = response.json()['access_token']
    token_string = '{0} {1}'.format(_token_type, _token)
    
    # I sue a session object, but you can also choose to directly input the headers into the .get/.post etc
    session = requests.session()
    session.headers = {
        'Authorization': token_string,
        'Accept': 'application/vnd+shotgun.api3_hash+json'
    }
    entity = 'projects'
    fields = ['name']
    filters = [['name', 'is', 'Start From Scratch']]
    url = urllib.parse.urljoin(shotgun_url, 'api/v1/entity/{0}/_search'.format(entity))
    query_response = session.post(url, data=json.dumps({'filters': filters, 'fields': fields}))
    
    if query_response.status_code == 200:
        print(query_response.json())
    else:
        #Seldom useful, returns messages like Not Found etc. But still for the sake of clarity printing to stdout
        print(query_response.reason)
        if hasattr(query_response, 'json'):
            # In case of a failed query, this should contain an error dictionary
            print(query_response.json())
        raise RuntimeError('Something went wrong during query, please check console for details.')

I mocked up a quick connect and query script. This will filter projects by name and return the name.
For assets, you could do a separate to query to the assets entity with [['project.id', 'is', <id>]]

Hope you find it useful.

Welcome to the community! :slight_smile:

6 Likes
4

Thank you very much for the help. this clears things up in some cases. Do you know how something like this would be written in javascript? For use on a website?

Please leave your original answer as it helps answer part of the question.

5

No problem, I’m glad it helped. For the JS part, since it is a REST API you can consume it the same as any standard REST API. axios is something I use all the time for HTTP requests.

It would involve setting up your API consumer with the necessary headers, for instance with axios please check this answer.

Off the top of my head, something like this:


import 'axios';


let shotgunUrl = 'https://somesite.shotgunstudio.com';
let loginUrl = `${shotgunUrl}/api/v1/auth/access_token`;
let authString = '';

const instance = axios.create()

instance.post(
    loginUrl, 
    {
        client_id: 'ascriptname', 
        client_secret: 'ascriptkey',
        grant_type: 'client_credentials'
    },
    options={
         headers: {
              Content-Type: 'application/x-www-form-urlencoded'
        }
)
.then(res => {
    authString = `${res.data.token_type} ${res.data.access_token}` 
})
.catch(err => {
      console.log('Auth Failed!');
      console.log(err);
})

if(!!authString) {
    // Set Auth header
    // Do queries
}

When the first auth promise resolves you should have a valid authString that can then be supplied to the Authorization like so instance.post.headers['Authorization'] = authString

It would depend entirely on your front-end how this fetch logic is handled.

Hope this helps.

4 Likes
7

Hey folks, awesome read here.

Just wanted to mention something you might already be aware of but wanted to call out for the broader community.

If you’re going to be running a query to find your projects you should opt for querying for your assets by filtering on the project itself rather than it’s name. A project is {} filter will be much more efficient than a project name is '' filter.

Cheers!

5 Likes
8

Thank you @bouchep for the tip!

I take it this applies to all aspects of the API? Such as if I have filters within tk-multi-loader2, the same optimization would apply?

2 Likes
9

Yeah, any time you already have an actual object (in Python it’s just a dict with the type and id) you’re better off using that instead of looking for the name of the entity through a link.

Sometimes, if you only have a name, it can even be beneficial to get the actual object using the name as the filter and only then doing your initial search with the object reference. Checking these scenarios is usually pretty quick and each set of filters is different so I suggest you test in your environment when you are developing your queries.

Also, the bigger the target dataset, the more likely searching for a linked name will be less efficient than searching with the object (Attachment vs Sequence, for example).

4 Likes
10

Could either of you explain why my query call shows a 400 code error in the console when i simply just try to get all projects using the samples above.

You can try it on my jsfiddle here https://jsfiddle.net/JokerMartini/oae28pnf/11/

1 Like
11

Hi @JokerMartini,


function queryShotgun() {
    var headers = {
        'Content-Type': 'application/x-www-form-urlencoded',
        'Accept': 'application/json'
    };

    $.ajax({
        url: 'https://shotgunUrl/api/v1/auth/access_token',
        method: 'post',

        headers: headers,
        data: {
            grant_type: "client_credentials",
            client_id: "ascriptname",
            client_secret: "ascriptsecret"
        },
        success: function (data) {
            console.log(JSON.stringify(data));
            console.log(data);
            $('#accessOutput').html(data.access_token);

            var headers = {
                'Content-Type': 'application/vnd+shotgun.api3_array+json',
                'Authorization': `Bearer ${data.access_token}`

            };
            var query = {
                    fields: ["name"],
                    filters: [["sg_status", "is", "Active"]]
                };
                

            $.ajax({
            		url: 'https://shotgunUrl/api/v1/entity/projects/_search',
                // url: 'https://studiodashboard.shotgunstudio.com/api/v1/entity/Task',
                method: 'post',
                headers: headers,
                data: `${JSON.stringify(query)}`,
                success: function (data) {
                    console.log(JSON.stringify(data));
                    $('#queryOutput').html(response.responseText);
                },
                error: err => {
                		console.log(err)
                    $('#queryOutput').html('Bummer: there was an error!');
                },
            })
        }
    })
}

window.onload = function(){
  queryShotgun();
};

Few things:

  • You need to do a post request when querying resources not get, this would again depend on the endpoint itself and the documentation will give you clarity on the supported methods for each endpoint.

  • The data needs to be a string representation. This means:

    var data = {
    filters: [["sg_status", "is", "Active"]],
    fields: ["name"]
}

    would be converted to string by using JSON.stringify(data).

  • When querying the Shotgun REST API, Content-Type must be one of

    • application/vnd+shotgun.api3_array+json or
    • application/vnd+shotgun.api3_hash+json
      i.e array or hash depending on whether you are querying for a list or a single object.

  • When attempting to query with arbitrary filters use the _search endpoint, not the resource endpoint, for example in the case of Project entity, the resource endpoint would be /api/v1/projects, a detail endpoint would be /api/v1/projects/PROJECT_ID and the filter endpoint would be /api/v1/projects/_search. Since you need a list of all active projects, this type of query falls under the _search criteria.

I’ve also changed the error function to an arrow function and logging out the err object to console. This error object would let you know about why specifically a query failed:

Example error object:

{readyState: 4, getResponseHeader: ƒ, getAllResponseHeaders: ƒ, setRequestHeader: ƒ, overrideMimeType: ƒ, …}readyState: 4getResponseHeader: ƒ ( key )getAllResponseHeaders: ƒ ()setRequestHeader: ƒ ( name, value )overrideMimeType: ƒ ( type )statusCode: ƒ ( map )abort: ƒ ( statusText )state: ƒ ()always: ƒ ()catch: ƒ ( fn )pipe: ƒ ( /* fnDone, fnFail, fnProgress */ )then: ƒ ( onFulfilled, onRejected, onProgress )promise: ƒ ( obj )progress: ƒ ()done: ƒ ()fail: ƒ ()responseText: "{"errors":[{"id":"2cceb987908dad656b85caece7815f80","status":415,"code":103,"title":"Unsupported Content-Type 'application/x-www-form-urlencoded; charset=UTF-8'","source":{"content_type":"Content-Type must be one of: 'application/vnd+shotgun.api3_array+json', 'application/vnd+shotgun.api3_hash+json'."},"detail":null,"meta":null}]}"responseJSON: {errors: Array(1)}status: 415statusText: "error"__proto__: Object

console.log(err.responseText.errors) will return a list of all errors. In this case 415: Unsupported Media Type... means that the Content-Type header was not set correctly,

Hope this helps, Good luck!

5 Likes
12

Thank you so much for your explanation and help. This is certainly helping me drastically understand how this works much better.

2 Likes
13

My pleasure, happy to help. :slight_smile:

2 Likes
15

@Sreenathan_Nair since you appear to be very knowledgeable and versed in the REST Api i posted another question here, taking into account your suggests and notes about the api. I’m not quite sure why im getting errors though. Hope you can help.

thanks

2 Likes