Sign a .rbz—requiring Trimble ID log in—from a script

#1

Hi everyone,

I’ve almost got this, but figured I might as well ask, since I still have a few steps left to figure out. (Also because identity.trimble.com is down right now, so I’m twiddling my thumbs…)

I’m trying to automate the extension signing process for our plugin, so I can integrate it into other scripts we use to build plugins for several platforms. Manual extension signing represents a break in the chain before we build our multiple-platform installers, so I’m trying to get a script for it that we can integrate.

We’re using Python and the Requests library, hoping to avoid having to do something like using Selenium (headless browser) to fill out forms.

Dealing with the .zip/.rbz/.hash file operations is no problem. The tough thing is making the http requests and getting through all the redirects and forms successfully, through the Trimble authorization service.

The process starts with https://extensions.sketchup.com/en/user/login, which also takes a destination param: https://extensions.sketchup.com/developer_center/extension_signature (where we ultimately want to end up). We use a GET request to get a hidden token from the page, and then make a POST with this token and our login data. This all seems ok so far, and we successfully follow a few redirects from here, but we don’t seem to get all the way through to being signed in—checking log in status by another GET request to the destination page (from a saved session object that stores cookies etc.) indicates that we aren’t signed in there.

I have a feeling we are missing something, indicated by the action tag associated with the sign in form. On that page, the action for the sign in form is given as ../../commonauth. Modeling from a successful manual sign in indicates that this expands to https://identity.trimble.com/commonauth. One confusing thing is that normally, using Requests for sign ins via POST means you feed the action as part of the request url, but in this case we already have a url, https://extensions.sketchup.com/en/user/login, so I’m not sure how to handle this.

At any rate, at this point I wondered whether anyone out there has tried something similar with the Trimble authorization. Thanks in advance for any thoughts you might have about this!

Jesse

#2

Not so much myself, but being able to automate the process would be very helpful, especially in a continuous integration setting. For example bumping the version number by clicking in a form field is superfluous as it could be specified in the extension itself (an extension manifest including all description, keywords & image references would save me also copy-pasting).

The current Extension Warehouse has not been built with a developer API (which now appears odd since it is oriented towards developers).

Ask @thomthom, he can either help figure it out, or take your input for what a developer API could look like.

#3

Thanks. Likewise, if others are interested in the resulting scripting for logging in to Trimble and posting/downloading .rbz files, I’ll post some relevant parts here.

In our case, we’re signing for an installer, so we have to do the whole extract-the-hash thing.

#4

We don’t support automation of signing right now. But it is on our radar. We need for perform some infrastructure changes first to support this.

#5

I was eventually able to sign in to Trimble ID from our script. Will post some steps once I get a little further along.

#6

Great to hear. Sorry for slow response on our end. We’ve been pretty busy these last days.

#7

Yes please do!

#8

The Trimble sign in is working, but I’m still working out how to get my .rbz post request working with that extension signing form. Will post some stuff once it’s further along.

#9

In particular, if anyone has any insight perhaps, I am trying to get through the file uploading process before I am able to submit the form for signing the extension. It is looking an awful lot like you’re forced to POST your file to the plupload uploader, which means you need to get a hidden token from the page. This is normally something you can do easily with a GET request, but in this case, all of pluploader is only rendered by JS after the page loads, meaning a GET request won’t catch it because it’s not there yet.

Anyone have any idea how I can get that token so I can POST to plupload, without resorting to a headless browser? I’ve been digging through the plupload JS, but haven’t turned anything up yet.

#10

have you tried from a HtmlDialog inside SU…

then you can use :execute_script for all sort of things…

john

#11

Interesting idea, but in this case this is one python script being called from another python script. Having it open SU might not be practical.

#12

An update:
In the end, I couldn’t find a way without a headless browser to get the JS-rendered plupload form for uploading .rbz files. I am currently using Selenium with geckodriver to run a headless Firefox browser for the purpose of the upload form. However, to minimize the time/overhead this adds, I’m not using that for anything else—so my Requests session passes its cookies over to the webdriver, which fills out the form and then hands control back over to my Requests session.

The sign in process is still entirely doable without the headless browser. Below is a method for signing in. (This is in Python, using the Requests library, as mentioned earlier…)

def cas_sign_in():
    """Use a POST request to sign in to the Trimble Developer Center
    using their CAS (Central Authentication Service).
    """
    destination = 'https://extensions.sketchup.com/developer_center/extension_signature'
    params = {'destination': destination}
    login_url = 'https://extensions.sketchup.com/en/user/login'
    action_url = 'https://identity.trimble.com/commonauth'
    username = <our actual username>
    password = <our actual password>

    # Initialize a requests session so we retain cookies and stay signed in.
    session = requests.session()

    # Use GET request to get the unique token from the sign in page.
    login = session.get(login_url, params=params)
    # Follow through 2 additional redirects to get the page with the complete form rendered on it.
    login = login.history[2]

    # Find the hidden fields on the page (including the token).
    login_html = lxml.html.fromstring(login.text)
    hidden_inputs = login_html.xpath(r'//form//input[@type="hidden"]')

    # Initialize the form_data dict with the hidden form values.
    form_data = {x.attrib["name"]: x.value for x in hidden_inputs}

    # Add our login data to the dict.
    form_data['username'] = username
    form_data['user'] = username
    form_data['password'] = password
    form_data['action'] = action_url

    # Log in
    session.post(
        action_url,
        data=form_data,
        params={'sessionDataKey': form_data['sessionDataKey']}
        )

    # # DEBUG: check the page using the logged in session to see if the url indicates we're signed in.
    # conf = session.get('https://extensions.sketchup.com/developer_center/extension_signature')
    # print conf.text

    return session
#13

Well, that didn’t take long—as far as I can tell, Trimble ID has changed their login process, rendering this obsolete. I’ll update it if I’m able to find a solution. The form here now also seems to be rendered by JS in the browser. That means that a normal request won’t get the form token, since the form won’t exist yet in the response.

#14

Ah, bummer. Yea the Trimble ID is out of the control of the SketchUp team.

Mind you, I have made a point of highlighting the need for Signing API internally. It would simplify our workflow as well. Kenny logged a formal request in the tracker: https://github.com/SketchUp/api-issue-tracker/issues/193 (might be worth following that thread if you are interested)

#15

Oh, that’s very good to know about, thanks.
Meanwhile, the solution here didn’t turn out to be complicated—the session’s request response turns out to have a history attribute, and the last element in this history array is the response after all the redirects. So by using the last element in the history, I’m able to get the form token and sign in again. I’ll update the code above.

I should add this is all still work in progress…