Developing Importer for .SHP (or SHX) to SketchUp

shp
import
shx
extension

#63

IMHO, a tool that doesn’t work for you isn’t much of a tool :wink:

There are several records in your dataset that have “bad” values. I’ve modified the plugin to skip those and note them in the Ruby Console log if they occur. I’ve renamed the plugin to spirix_shapefile_importer to differentiate it from the previous version. It seems to digest your data using a scale factor of “1.0” and “buildinghe” for the database height, but skips 83 records. The new version is available here.

[added]

It was previously suggested that the bounding box values be subtracted from the co-ordinate data prior to scaling. This improves the dynamic range. but loses the relative reference points if importing “tiles” of datasets. However, in this case, no amount of juggling the scale factor worked with the range so the imported data is now always referenced at the origin (lower-left corner).

The data could be in furlongs or cubits or somesuch … it’s all relative and dependent on the dataset. When you’ve finished importing the buildings, you will need to scale the model to match the real-world units.


#64

Maybe a nice option to add to the import parameters? [input units source file] and have the script convert the data to the current model units?


#65

I think that’s a good suggestion. The ideal solution would be to auto-scale and geolocate the model when importing. However, when testing various datasets from around the world, I found that the point data was all over the map (no pun intended). The two basic types were uncorrected (raw) and corrected data. In order to correctly interpret the numeric information in the dataset, several things must be known … the coordinate system used, the projection used (if any), the scale factor used, the datum used (horizontal, vertical, etc.), as well as the accuracy and precision of the numbers themselves. The developers of the various datasets I looked at used whatever was convenient to them in assembling their shapefiles … they may or may not have adhered to one of the several popular GIS standards. Some appear to have invented their own. As I noted above, the units could be in furlongs for all I know. Given that there was no reliable way to auto-magically determine what the numbers actually mean in every case, I decided to just use the numbers as raw coordinates and provide an optional scale factor to deal with tiny geometry. Not only was it much easier to do so, of course, but it means that the plugin will work with other types of shapefiles as well.


#66

Hi,

I’m new here, so maybe i did something wrong, but the the zipfile “https://data.cityofnewyork.us/download/tb92-6tj8/application/zip” seems to be corrupted or something, it can’t be extracted. FIles inside are named “buinding_1015” so i suppose this is a new version of the database. Do you know where I can found your shapefiles version ?

Thanks


#67

I’m not sure what might be causing you to have problems. I downloaded the zipfile and it seems to be okay. It’s 165,098,495 bytes in size and has 11 files. The “building_1015” naming corresponds to the timestamp on the files as 10/28/2015. I was able to open the zipfile and extract the “building_1015.shp”, “building_1015.dbf”, and building_1015.shx" files to create the first thousand-odd buildings.

The latest version of the plugin can be found here:

https://sites.google.com/site/spirixcode/code/spirix_shapefile_importer.rbz


#68

hummm … mine is only 65 Mb, It seems that something goes wrong when I download it. I’ll try tomorrow with an other connexion.

thanks


#69

I keep getting a warning "blocked by SmartScreen Filter. Is there something we are not being told about this file?


#70

Which file? The zipfile or the plugin? Does the warning give any details?


#71

When I select the .rbz link you provided I get the warning.

The latest version of the plugin can be found here:

https://sites.google.com/site/spirixcode/code/spirix_shapefile_importer.rbz


#72

Does the warning give any specifics? Microsoft notes that:

SmartScreen Filter checks files that you download from the web against a list of reported malicious software sites and programs known to be unsafe. If it finds a match, SmartScreen Filter will warn you that the download has been blocked for your safety. SmartScreen Filter also checks the files that you download against a list of files that are well known and downloaded by many Internet Explorer users. If the file that you’re downloading isn’t on that list, SmartScreen Filter will warn you.

and

When you download a program from the Internet, SmartScreen Filter will check the program against a list of programs that are downloaded by a significant number of other Internet Explorer users and a list of programs that are known to be unsafe. If the program you’re downloading isn’t on either list, SmartScreen Filter will display a warning that the file isn’t “commonly downloaded.” It doesn’t necessarily mean the website is fraudulent or that the program is malware, but you probably shouldn’t download or install the program unless you trust the website and the publisher.

The plugin is a Windows zipfile that has been renamed from “spirix_shapefile_importer.zip” to “spirix_shapefile_importer.rbz” … the only thing inside the archive is “spirix_shapefile_importer.rb” which is a Ruby text file that you can read. You can either instruct SmartScreen to go ahead and let you proceed, or you can try the unzipped version here:

https://sites.google.com/site/spirixcode/code/spirix_shapefile_importer.rb


Importing Google Earth Data. How to?
#73

hi,
can you test if you import these shape file ??

i’ve just installed the ultimate version of your tool but i don’t see nothing in sketchup when i import the file…


#74

If you open the Ruby console (Window | Ruby Console) before running this, it will give an error message indicating that there are no polygons in the shapefiles you have.

C:\Users\Username\Documents\SketchUp\POZZO/POZZO*.shx does not contain polygons!

From the name of your zipfile, am I to assume that you are looking for shapefiles from the town by the same name in Italy?


#75

yes yes… i am in Italy…
the shape file was created with qgis 2.2 and it works fine …
why sketchup doesn’t import it ?


#76

Just to clarify, this is not a problem with SketchUp … it’s a problem with a third-party plugin.

The plugin is designed to import type 5 shapefile Polygons and ignores any other types. Your shapefile contains type 15 polygons (PolygonZ). However, the PolygonZ record format is an extension of the Polygon record format and I was able to import it by modifying the plugin to accept type 15 instead. Attached is the result of importing your file … you will need to locate and scale it to fit your needs:

porto_pozzo_1.skp (65.1 KB)

[added] I belatedly noted there is height data available:

Here’s a model with the same data but includes the building heights:

porto_pozzo_1a.skp (119.7 KB)

You can also download shapefile data from http://download.geofabrik.de/europe/italy/isole-latest.shp.zip (which contains 986,411 building footprints). This has data for the highlighted region:

Using the following values, you can limit the import to just the Porto Pozzo region:

porto_pozzo_2.skp (146.5 KB)

There is also Open Street Map data available from http://www.openstreetmap.org/ (which contains roads, coastlines, etc., in addition to buildings):

porto_pozzo_3.skp (408.9 KB)

If you have an ongoing need to import more data from your QGIS application, let me know and I will update the import plugin to accommodate the PolygonZ (and PolygonM) format as well.

[added 2]

Here’s another version that has all three imports at a nominal scale in separate layers:

porto_pozzo.skp (657.0 KB)


#77

hi,
thanks a lot !!!
but the shape doesn’t contains height…

i have to import shape into sketchup from qgis beacause i create it with height.
in a second moment i have to import a dxf file (electromagnetic field) in the skp with my shape file and export all in a 3d pdf…

now i do all of it but i can’t import my shape file…

I’d be grateful if you could give me your plug modifyed !!!


#78

I’ve uploaded the new version here … let me know if you have any problems, etc.


#79

hi,thanks a lot…
i have downloaded and copyed in the plugin directory the new file but i can’t see it in the sketchup extension…

may you help me please ?


#80

If you downloaded the spirix_shapefile_importer.rbz file, you should use the “Window | Preferences | Extensions” panel and the “Install Extension…” button to select the file you downloaded:

This installs the plugin for you … no need to copy it to the plugin folder.


#81

ok i have done it…
but i can set the same height value for all buildings…
sure I’m doing something wrong…

Solved !!!

But when a bulding has the foundation (piede) under the value “0” sketchup does not represent it…(red color)


#82

I’m not exactly clear what you are trying to do … can you give a little more information? If you could create two buildings in QGIS and then post the zipfile here along with the dimensions (and height) that you think they should be, I can try to modify the plugin to do what you need.

It’s possible (or even likely) that the height data is in the extended values of the PolygonZ data and not in the associated database file:

PolygonZ
{
  Double[4] Box // Bounding Box
  Integer NumParts // Number of Parts
  Integer NumPoints // Total Number of Points
  Integer[NumParts] Parts // Index to First Point in Part
  Point[NumPoints] Points // Points for All Parts
  Double[2] Z Range // Bounding Z Range
  Double[NumPoints] Z Array // Z Values for All Points
  Double[2] M Range // Bounding Measure Range
  Double[NumPoints] M Array // Measures
}

The last four parameters may be holding your height data … posting a simple example will help me figure out what’s what.

The basic instructions are in the header of the plugin … I’m posting them here in case you are unsure how to open the plugin for editing:

# Two files are needed to properly import a shapefile:
#
#    example.shp (shapefile containing polygon information)
#    example.shx (shapefile containing index information)
#
# A third file is necessary if height data needs to be added to the 2D shapes:
#
#    example.dbf (dBase file containing attribute data)
#
# The importer currently only imports type 5 shapefiles which contain polygon information
# with (possibly) multiple parts. Wherever more than one part is encountered, it is added
# as a hole in the parent polygon. Each polygon is grouped and named with the index value
# appended to the specified name (if no name is specified, only the index number will be
# used). The default name is the time that the importer was launched in the format of:
#
#   YYYYMMDDHHMMSS
#
# The default scale is set to 1.0, however, if the program crashes with a too few points
# error, it can be increased as needed. Inspection of the bounding box values compared to
# the real-world size of the box should provide a rough guideline on setting a proper
# scale factor. Note that the initial bounding box values are taken from the shapefile
# itself and are not scaled.
#
# A good approach to large datasets is to import only the first 1000 records or so and
# then overlay a map to determine the bounding box you want to use for all the records.
#
# TIP: before starting the import, create a short line at the origin to use with the
#      tape measure tool to determine the co-ordinates of the bounding box.
#
# 3D height data is available in associated example.dbf files, but each requires a
# different schema map and field names to match. However, in lieu of that, each imported
# polygon can have an optional height applied to it by entering the height where
# indicated (100.0 or so is a good start; you can always scale the groups when done with
# the import).
#
# Larger imports may consume more than the physical memory available. While the import
# will probably finish someday, it could take figuratively forever. In this case, the
# following two lines should be commented out where they appear in the code:
#
#        @@model.start_operation("Import Shapefile",true)
#        @@model.commit_operation
#
# This will run somewhat slower, but it will finish in a reasonable amount of time.
#
# Shapefiles for most countries and states can be found at:
#
#      http://www.geofabrik.de/data/shapefiles.html
#
# Click on the link to download shapefiles. Unzip in a folder and then launch the
# importer, browse to the *.shp or *.shx file, and select one of them (either will
# do - however, both file types need to be available in the same folder).
#
# Version 3.0a introduces database support for building heights. Once the files have been
# selected, choose "Yes" on the dropdown for Database and on the next screen select the
# field from the choices. The height value should work with any numeric format. All
# heights will be scaled by the overall scale factor. If the heights are not proportional
# simply select all the groups and use the scale tool to adjust the z height.
#
# No smoothing is performed, but once the buildings are created, select all of the groups
# at once and apply the smoothing tool to any one of them to smooth all of them.
#
# Files obtained from:
#
#      https://data.cityofnewyork.us/download/tb92-6tj8/application/zip
#
# will work with the "HEIGHT_ROO" value from the associated database.
# Files obtained from:
#
#     http://egis3.lacounty.gov/dataportal/2011/04/28/countywide-building-outlines/
#
# will work with the "HEIGHT" value from the associated database. Other datasets that
# provide height values will usually be noted in their schema dictionary.
#