Here’s my revised cas_sign_in() function, all working again. Obviously this is somewhat sanitized, and I’ve left out my logging. However, I still can’t figure out why I’m not getting a .hash file (only a .susig) in my download link, at the end of the new extension posting function.
def cas_sign_in():
"""Use a POST request to sign in to the Trimble Developer Center
using their CAS (Central Authentication Service).
"""
destination = 'https://extensions.sketchup.com/extension/sign'
params = {'destination': destination}
login_url = 'https://login.sketchup.com/login/trimbleid'
action_url = 'https://identity.trimble.com/commonauth'
username = <your user name>
password = <your password>
# Initialize a requests session so we retain cookies and stay signed in.
try:
session = requests.session()
# Use GET request to get the unique token from the sign in page.
login = session.get(login_url, params=params)
# Follow redirects to finally get the page with the complete form rendered on it.
login = login.history[-1]
# Find the hidden fields on the page (including the token).
login_html = lxml.html.fromstring(login.text)
hidden_inputs = login_html.xpath(r'//form//input[@type="hidden"]')
# Initialize the form_data dict with the hidden form values.
form_data = {x.attrib["name"]: x.value for x in hidden_inputs}
# Add our login data to the dict.
form_data['username'] = username
form_data['user'] = username
form_data['password'] = password
form_data['action'] = action_url
# Log in
res = session.post(
action_url,
data=form_data,
params={'sessionDataKey': form_data['sessionDataKey']}
)
# Confirm login by requesting identity info from the API
api_me = 'https://extensions.sketchup.com/warehouse/v1.0/users/me'
me_json = session.get(api_me).content
me_data = json.loads(me_json)
# The JSON contains other fields, such as an ID number, which we could
# also use for this verification.
if <your display name> not in me_data['displayName'].lower():
raise Exception("Login failed. Please check login credentials.")
return session
except Exception as e:
raise
And here is the revised function for uploading, signing, and downloading the .rbz using the new API—but this is not completely working. I’m posting it here in the hopes someone may have some insight. The signing API returns an ‘Extension signed successfully’ message, and gives a working download link, but for some reason, the results I get from the script are always missing the .hash file, even though they include the .susig file. Anyone have any ideas?
(N.B.: I’ve removed my logging around exception handling. You might want to add some print statements or other logging there when you use this. Also, you’ll need your signed-in session from the sign in function above in order to make these requests.)
def post_rbz(rbz_file, session):
"""Use our signed-in session to send POST requests to SketchUp's extension
signature API. There are two POST requests required to to make this work:
POST 1: upload the .rbz file to the upload endpoint.
POST 2: make a request to the .rbz signing endpoint, including the new
uniquely-generated name reference to the uploaded .rbz file.
After posting, get the download link from the response, and then pass it
back to our session to download it. Return the signed file.
"""
try:
files = {('file', open(rbz_file, 'rb'))}
# # DEBUG: use this hook callback function to check the POST response.
# # To use this, enable this line in the POST request:
# # hooks={'response': print_content}
# def print_content(r, *args, **kwargs):
# print(r.content)
api_url = 'https://extensions.sketchup.com/api/v1/extension/upload'
res = session.post(
api_url,
files=files,
# hooks={'response': print_content} # enable for debugging
)
except Exception as e:
raise
try:
# Get the reference name that was generated for the uploaded rbz file.
extension_rbz_unique = json.loads(res.content)["rbzFileNameUnique"]
# Set up the signing data to submit, with references to our uploaded
# file. If you want the results scrambled or encrypted, change False to True.
form_data = {
'scramble': False,
'encrypt': False,
'extensionRbzUnique': extension_rbz_unique,
'extensionRbz': '<your extension name>.rbz'
}
signing_url = 'https://extensions.sketchup.com/api/v1/extension/sign'
# Post to the API signing endpoint.
res = session.post(
signing_url,
json=form_data
)
except Exception as e:
raise
# Parse the JSON response to get the download link.
try:
res_json = json.loads(res.content)
dl_url = res_json['extension']
except Exception as e:
raise
try:
# Use the session to download the signed .rbz extension.
temp_dir = os.path.dirname(rbz_file)
dl_rbz = os.path.expanduser(os.path.join(temp_dir, '<your extension name>.rbz'))
res = session.get(dl_url, stream=True)
with open(dl_rbz, 'wb') as f:
shutil.copyfileobj(res.raw, f)
return dl_rbz
except Exception as e:
raise
Follow-up question: if signing no longer produces .hash files, does this mean that it is no longer possible to release new plugin versions for SU 2016 with a signature? We still currently support back to SU 2014 with our new versions. It looks like, with just a .susig file, they’ll show as signed in 2017-2020 but not 2016.
Hey all—on @tt_su 's suggestion I just checked my script to make sure the Trimble ID sign in still works. (@tt_su mentioned there may have been recent changes to the sign in.)
I’m happy to report it still works!
If anyone wants further details or help about how to sign extensions this way via script, feel free to ask.
Traceback (most recent call last):
File “/builds/tsk/3skeng-install/src/sign.py”, line 195, in
sketchup extension signing
starting session with Trimble Identity
main(sys.argv[1:])
File “/builds/tsk/3skeng-install/src/sign.py”, line 188, in main
session = cas_sign_in(username, password)
File “/builds/tsk/3skeng-install/src/sign.py”, line 38, in cas_sign_in
login_html = html.fromstring(login.text)
File “/usr/lib/python3.9/site-packages/lxml/html/init.py”, line 875, in fromstring
doc = document_fromstring(html, parser=parser, base_url=base_url, **kw)
File “/usr/lib/python3.9/site-packages/lxml/html/init.py”, line 763, in document_fromstring
raise etree.ParserError(
lxml.etree.ParserError: Document is empty
Hm. So the error is being raised by ETree, when it tries to parse the hidden fields on the login form (from which you used to need to get a token). As far as I can tell, there no longer are any hidden fields, so we can get rid of those lines. I would guess the credentials are being passed through by some other method. I haven’t been able to confirm what this method is yet, so I am stuck getting error messages from my POST request. Incidentally, it seems like the action_url value (where the form should actually get POSTed to) has changed to 'https://id.trimble.com/ui/api'.
It’s possible this is related to the state value on the page, which you can follow from watching the chain of requests and responses in a browser. When I pass this through to the new action_url, I get a 502 Bad Gateway error, so either my state token isn’t doing what I think it is, or I’m not using it correctly. I could be off-track here—please take a look if you think you might be able to help. Otherwise, I’ll be back at it tomorrow.
All right, after more investigation, it looks like what’s happening is that our login form POST requires some Optanon cookies. I’m not seeing that the browser ever gets these cookies in any of the redirect responses—they just appear in a request, which makes me wonder whether some JS on the page isn’t injecting them. This wouldn’t be a problem if our session could pick them up during the redirect chain, but I was also finding that this chain wasn’t getting to this point (the API request) by itself—I needed to make the last GET request manually in order to supply the correct URL. Still working on it…
I’m using this script and I had to modify the sign in portion to get it to work. With a few changes I have my deployment script back up and running. My computer crashed so I can’t post code at the moment.
Hey folks, glad some of you have found other solutions. In our case, it is far preferable to be able to do the signing without running a headless browser, otherwise I would’ve gladly made that move—and I do recommend it if it works for you.
I did learn a little bit more about the cookie-based authentication. In order to get his Optanon cookie for the POST request, the page calls into a CDN at cookielaw.org, which fetches a dense, not-very-human-readable JS function. Running the function gets you some JSON which seems like it represents the cookie somehow, and this is then added to the headers in the following request. I had not quite connected every dot here yet, and there is the possibility that there’s some other security that ultimately will make this a dead end.
In order to get a SketchUp 2022-compatible release out quickly, we’re going to just sign manually for this round. Afterwards, we still have some ideas to look into for getting this to work, and I will of course update here if I have something.