Developing Importer for .SHP (or SHX) to SketchUp

shp
import
shx
extension

#1

I have been working on creating something similar (just for the sake of learning – im a college student) to these maps/renders that really inspired me.

https://m2.behance.net/rendition/pm/24276859/max_1200/e7d521c53f6abd7c51051fb9cff6ecaa.jpg

While this particularly amazing example is definitely not using sketchup, I have seen examples to this made in sketchup as well – so i know its possible. One can be found on the sketchup home page currently.

Here is where I am stuck:

  • I have a shapefile that defines each building’s perimeter and includes an attibute for hieght, so all the information to create a simple 3d model of all the buildings is there.
  • I am stuck with how to make sketchup accept this information in a quick and efficient way. Maybe some kind of ‘data/attribute defined’ push pull tool applied to all the polygons? Is something like that possible?

Or is this kind of ‘communication’ between file types/software not feasible?
I am using Qgis and ArcGIS for the GIS end, and obviously sketchup for the 3D side. I am avoiding autoCAD because my access is limited as this is something i am figuring out in my own time – on my own computer and software.


Finding Context Models for City Planning
#2

Are you referring to ESRI-style shapefiles? Do you have a small sample shapefile you can share here? I think if you rename something like “test.shp” to “test.zip” it will upload to the forum (I assume it’s actually a binary file).


#3

Why not?


#4

I don’t see why it couldn’t be done in SketchUp (it might even look better). However, this particular model was done by Luis Dilger who says “The OpenStreetMap data enabled me to visualize the satellite-based information using DEM Earth in Cinema 4D.”

BTW, Google Earth has done a remarkable job of combining angled satellite photos with vertical satellite images along with even more detail from their street-view mappings. Here’s a random shot of a town showing the 3D effects. It’s not until you look closely at the trees and cars that you start to notice the loss of the illusion.


#5

Because the designer said he used cinema 4D :wink:


#6

Software requires an API for automation. SketchUp has both a Ruby scripting API, and a C API (work in progress.)


#7

That sounds like it would be a perfect job for a script.

I made something similar to that a few years ago in my old job: Architect Tools

I often got a lot of DWG files where I needed to recreate the buildings. I found that I could extract the 3d data into making the base block shapes:

I don’t know if you are script savy or not - but given a file that describe the perimeter and height of your buildings it shouldn’t be too hard to generate some blocks from that.

If you post a sample source file here there might be someone around here who would be able to assist.


#8

In researching this, I came across some sample files. In order to post them to the forum, I’ve changed the names to use a zipfile extension:

THESE ARE NOT ZIPFILES!

5961_shp.zip (1.5 MB) (Geometry: rename to 5961.shp)

5961_shx.zip (1.7 KB) (Index: rename to 5961.shx)

5961_dbf.zip (120.4 KB) (Metadata: rename 5961.dbf)

ESRI Shapefile format specifications:

shapefile.pdf (123.8 KB)

[edited] Info about OSM files from GEOFABRIK: http://www.geofabrik.de/data/shapefiles.html

Daniel Azuma has written a Ruby Gemfile module for reading the files:

Not an easy importer to write from scratch … but it looks like most of the heavy lifting has already been done :wink:


How to extract 3d-building from google earth 2017 to sketchup
#9

I cobbled together a simple script to read a shapefile. It only creates polygons which are typical of what is found in a file called “buildings.shp”. I have not yet cross-referenced the corresponding “buildings.shx” and “buildings.dbf” so there is no height information available … just 2D polygons. The geometry looks “skewed” slightly, but it matches Google Map’s satellite view so I think maybe some sort of geospatial correction needs to be applied. There is also a “tiny edge” problem even though I subtract the bounding box X and Y minimums and scale by 1000x to try to increase the dynamic range of the data. My attempt to import the city of Dresden resulted in this:

The area indicated by the red rectangle maps to the satellite view:

This is one of the few times I really wish I had a faster computer …

Sample script: jimhami42_import_shapefile.rb (1.9 KB)


#10

After RTFMing the ESRI, Ruby, and SketchUp APIs (again), I finally came up with this:

jimhami42_shapefile_importer.rb (3.7 KB)

It uses hard-coded filenames in the “c:\temp” directory for simplicity while I’m still playing around with it. I imported all of the buildings from Hawaii but the .skp file is too big to post here. However, here’s a sampling of downtown Honolulu:

oahu_partial.skp (2.0 MB)

It only processes polygons and, of these, only those with single parts (more than one are skipped). I’ve tested it under both SketchUp 8 and SketchUp 2015 Make.

[edited] I’m not sure why I’m having problems this morning, but I’ve replaced the plugin and the sample model several times and screwed it up each time … I think I finally have it straight :frowning:


#11

(Moved to Developers category, edited topic title & tagged.)


#12

Wow. Thank you sir! You really went above and beyond for me. I have found a
few scripts that do similar things… I just haven’t been working on this
much in the past few days as school is beginning to take over my time again
and summer is over. But I totally agree – my rendering time in QGIS with a
.shp file for every building in LA loaded in was atrocious and is why i
didnt upload a shp file sample haha – I cant wait to eventually get a
super powerful machine that doesn’t even blink at these file sizes.

I will try your script out soon, and let you know how the other scripts I
found work out. (this one seems the most promising:
http://anitagraser.com/2014/03/15/3d-viz-with-qgis-three-js/ ). I would
love to get an awesome finished look out of this, but I think my best move
is just to take it slow and tinker. What would you recommend as a resource
to begin learning Ruby? (that is the language right?) As other commenters
and your method point out, that is probably the best method to make this
work and I do have a bit of coding experience in R, SQL, and python. There
is no point this little project, just want to increase my knowledge outside
of what is provided to me at school – Im a GIS/Geography student at UCLA
and they sadly don’t offer much on the 3D modeling side of the field.
Thanks again for your time, have a good weekend
Rainer


#14

Not really … SketchUp is my hobby time. The hardest part about my hobby is finding something useful to do using SketchUp :wink:

I’m the wrong person to ask about this. I’ve got very limited experience with Ruby and produce mostly hacky code that causes the more experienced programmers to gnash their teeth. I found this helpful for Ruby: http://ruby-doc.org/ and this for SketchUp: http://www.sketchup.com/intl/en/developer/.

I’ve added names to the polygons as text tags from the database, but I used a somewhat kludgy approach. As long as you use OSM shapefiles for buildings, I think it will work okay. Again, the plugin expects files to be in the “c:\temp” folder.

New version (1.0a): jimhami42_shapefile_importer.rb (4.5 KB)

Here’s an sample model using data for Hawaii from http://download.geofabrik.de/ (I had to pare it down some to just Honolulu to keep the file size reasonable):

honolulu_part.skp (1.9 MB)

My initial thoughts before diving into this was that it would be more suited to C instead of Ruby, but the entire state of Hawaii (about 14,000 buildings) took just under a minute to process on my wife’s computer which has an “Intel® Core™ i3-3220 CPU @ 3.30GHz” (whatever that is) with 4.00 GB of RAM. Not all that bad, actually.

Sad to say, the only fields supported in the “buildings.dbf” file are “osm_id”, “name”, and “type”. There is no height data :frowning: If you have a sample DBF file that has height data that I could take a look at, I might be able to apply the push / pull to the polygons (which, if I recall, was your original question). Or not.


#15

The link you provided showed a layout using constant height for the buildings. I imported the state of Hawaii and extruded the buildings a constant height to see how that would look. If they had a name in the database, I set this as the name of the group for the building. The result is about 35.6 MB which is small enough for the warehouse:

https://3dwarehouse.sketchup.com/model.html?id=ubb9d7b9f-9518-445b-8ae5-d8b3c4a1dd17

My day job has been really busy this week, so I haven’t had a lot of time to play with this. However, I plan on diving into this veritable cornucopia of information this weekend:

geofabrik-osm-gis-standard-0.6.pdf (165.0 KB)

I can see a lot of fun stuff just around the bend in the road :wink:

[added] ----------------

[more added] -----------------
Rendered as 3D printed in Thea (free version)

Native view in SketchUp:

Ready for printing:


#16

See other topics in Ruby API sub-category:

search category:Ruby API for “Learning Ruby”


#17

From what I’ve learned, there are basically three files that make up a “shapefile”:

  • *.shp - contains basic objects (geometry, etc.)
  • *.shx - index for *.shp (and/or *.dbf)
  • *.dbf - optional attribute file

The only Z information in a *.shp file appears to be elevation data (both estimated and measured). If a polygon in the *.shp file is a building footprint, its height, if there is one, will be in the database. I believe that the code I posted previously will work will most *.shp and *.shx files as is. However, I found that the *.dbf files vary widely depending on the source of the dataset. For Open Street Map derived shapefiles from GEOFABRIK, for example, they generally list only the name and type. For New York City, on the other hand, they provide their own format which includes height data (among other attributes mostly relating to property tax assessment). By using the same *.shp and *.shx access, I only needed to modify a few lines of code to incorporate the height from their *.dbf file:

There’s over a million buildings in the database which might easily be handled by someone’s else’s computer, but certainly not my wife’s. On top of that, I was a bit surprised to see that some of the buildings are made up of hundreds of items. I ended up settling on West Manhattan and skipped over any buildings that weren’t in its bounding box (26,566 buildings are shown above). I also left off any buildings that had no height listed. Here’s another render using my free copy of Thea:

Native SketchUp mode:

Ready for printing:

I will post a new version of the code soon … I’ve got it crudely working for 1, 2, or 3 parts per polygon, but some of the buildings in the NYC dataset had 8 or 9 parts. I have a final approach for that before it can be released into the wild … I’m not sure how to handle the various *.dbf files that may be encountered ;(

BTW, if you can split off the first 20KB of your *.dbf file and post it or send it to me, I’ll be more than happy to tailor the database part of the code to match.


#18

Virtually playing around with Manhattan Island can be really fun :smile:

15,310 buildings (those without heights are colored red)


#19

I managed to get your script to run on my Mac, but the model looked different to your screenshots.


#20

change model units to some draconian imperial format and it might work…


#21

I was on feet/inches, I’ve changes to metres, see if that helps. It takes a while to do the import!