I am just having a look at ImageRep, but I don’t see a method to convert a Sketchup:Color object (or a 4-uple [R, G, B, Alpha] into a 24-bits data required for PNG.
Say that I want to replace all pixels in Black color by White color.
you can just modify the bytes string and then feed it back modified…
image_rep = Sketchup::ImageRep.new
image_rep.load_file(in_file)
# @params
bits_per_pixel = image_rep.bits_per_pixel
row_padding = image_rep.row_padding
width = image_rep.width
height = image_rep.height
size = image_rep.size
# get the raw bytes
raw_bytes = image_rep.data
# work with non binary string to avoid crash
bytes_string = raw_bytes.bytes
raw_bytes.clear
image_rep = nil
# run_gc
# return 3 or 4 bits for most images...
bits = bits_per_pixel/8
# crude test for original B or W transparency
mode = bytes_string.first(bits).sum/bits
# slice into chunks - all bytes in row [pixels and padding]
rows = bytes_string.each_slice(size / height).to_a
# pixels
rows.each do |row|
pixels = row.each_slice(bits).to_a
pixels.each do |byt|
avg = byt.sum/bits
if avg != mode
# modify each bty in here
end
# new_ary << byt
end
end # chunks
john
Ok. What I suspected… NO simple solution.
ImageRep has a method to extract the colors (imgrep.colors). At least this avoids to go down to the nitty-gritty of bytes and bits manipulation.
It would have been simpler to get also the dual method to encode a color [R, G, B, A] and put it back in the colors array (with a method like #set_colors(color_or_array) or #replace_colors(index, color_or_array).
After all, an image is finally just an array of color specifications.
:image_rep.colors is very expensive…
it creates a new Sketchup:Color for every pixel…
so for an image with only 64 unique colours it creates [width * height *[ Sketchup:Color.new]] ,which can be thousands…
you then need to turn them back to arrays and sort into a hash, just to get the unique colours…
it’s easier to deal with byte_string, which are in RGBA format anyway…
this is one of the rows after :each_slice(4) from the example above…
row = [[0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [193, 70, 70, 255], [219, 145, 145, 255], [193, 70, 70, 255], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0], [193, 70, 70, 255], [219, 145, 145, 255], [193, 70, 70, 255], [0, 0, 0, 0], [0, 0, 0, 0], [0, 0, 0, 0]]
john
1 Like
I thought that was straight, but I guess that if you encode 4 values R, G, B and Alpha, into 24 bits, there must be some compression tweak somewhere.
Note also that, on a PNG image of 256 x 256, encoded in 24 bits, it takes less than 1 ms to create an ImageRep from the PNG and loop on all colors.
So performance are not so bad.
it fine for a one off, but they all stay in memory, and are harder to purge than arrays and hashes…
I’m trying to use it for conversion to vectors so I need to be very careful about memory build up…
it’s a shame @ChrisFullmer doesn’t sort out exposing more of ‘open source’ FreeImage library to the API…
john
I am fine with manipulating data. That’s not the issue.
But how do you convert a sequence of 4 element-tuple [R, G, B, A] in 24 bits elements?
Yea, I experienced the same when I was updating Bitmap2Mesh to use ImageRep.
I have a module in that project that might be useful. In particular, it also deals with the platform dependant order of the color bytes:
I also ended up doing my (incomplete) material clone method: https://github.com/thomthom/bitmap-2-mesh/blob/master/src/tt_bitmap2mesh/helpers/image.rb#L14
This reminds me that I should have reported a number of these in the issue tracker (GitHub - SketchUp/api-issue-tracker: Public issue tracker for the SketchUp and LayOut's APIs).
There are some API improvements that would make life easier. Some documentation improvement.
Exposing FreeImage directly wouldn’t be good API design. FreeImage is just an implementation detail.
ImageRep was not specially expected nor requested (as far as I know) in SU2018. But nothing is worse than a half-baked API. So please discuss it with developers beforehands.
My analysis is that, precisely in the case of ImageRep, performance counts a lot, and thus all bits and bytes manipulations should be done in C/C++ under the cover to deliver a usable API to developers, typically with high-level methods such as:
- #crop
- #replace_color { condition }
- #grayscale
- etc…
Going through millions of bytes in Ruby loops and processing is certainly not efficient, furthermore when the code depends on platforms and other low-level considerations. In addition, what a wasted effort, because every developer will spend time on his own implementation of the same functionality.
And for material.copy(), it would have been better to introduce this method in the API, whatever is the way it’s implemented under the hood. Again, for performance reasons and also because developers prefer to put their focus on the functionality of their script, not on troubleshooting ancillary problems.
My take is that ImageRep is simply not usable the way it is delivered. It’s a pity, because it would have had a big potential. And frankly, I don’t see any reason for this ‘half-baked’ delivery. If it’s a matter of time, then just release it in a next version of Sketchup, but at least with something usable, and also documented.
As a suggestion, and to limit the damage, it would be helpful that someone from Trimble publishes a snippet of the Ruby code for the high-level methods mentioned above (crop, substitute colors, grayscale).
1 Like
I finally managed to implement a substitute_color() method, which allows to change the colors of the pixels of any PNG based on a specified substitution method taking a (r, g, b, a) in input and returning a modified (r, g, b, a) in output. This is actually a few lines of code.
What had puzzled me (and made me railing) is my lack of understanding of the PNG format about transparency for PNG files encoded in 24 bits, that is, how to fits 4 8-bits integers in 24 bits…!!
Actually, the 24 bits are for R, G, B values (each on 8 bits). The transparency is encoded in the PNG file separately, since it applies uniformly to all pixels. It’s only with 32-bits PNG that you can have individual opacity per pixel.
The method is reasonably fast. I get around 70 ms on a 256 x 256 file.
# Substitute colors in a PNG file <file_img>
# If the output image file is not specified, the input file is replaced
# Method substitute_proc takes arguments (r, g, b, a) and returns a new [r, g, b, a]
def image_rep_substitute_colors(file_img, file_out=nil, &substitute_proc)
imgrep = Sketchup::ImageRep.new(file_img)
file_out = file_img unless file_out
#Data bytes are processed by pixel as [r, g, b, a] (Mac) or [b, g, r, a] (Windows)
#<a> is missing for 24 bits files
new_data = ""
if RUBY_PLATFORM =~ /darwin/
imgrep.data.bytes.each_slice(imgrep.bits_per_pixel / 8) do |pixel|
r, g, b, a = substitute_proc.call(*pixel)
new_data << r.chr << g.chr << b.chr
new_data << a.chr if a
end
else
imgrep.data.bytes.each_slice(imgrep.bits_per_pixel / 8) do |pixel|
r, g, b, a = substitute_proc.call(*pixel.reverse)
new_data << b.chr << g.chr << r.chr
new_data << a.chr if a
end
end
#Saving the data in to the output image file
imgrep.set_data(imgrep.width, imgrep.height, 8 * new_data.length / imgrep.width / imgrep.height,
imgrep.row_padding, new_data)
imgrep.save_file(file_out)
endImageRep came from requests of being able to extract image data without writing to disk. Main source of this was exporters and visualising/renders. These often use the C API to read the model, so ImageRep was first added there and the main intent was to be able to read data.
We also had a good use case for reading image data in the Ruby API from Modelur. Jernej presented at an earlier DevCamp a concept where if we could read pixel data from a texture or image he could use the color and map it to zoning regulation which drove the building height etc.
So we added Ruby API support for ImageRep in several places of the API. Among them being able to sample based on UV coordinates.
Providing read/write of rather low level data like this was a fairly easy thing to do.
We did think we might get requests to manipulate images after exposing ImageRep. But we did not go further that release because we didn’t have as solid set of use cases and requests at the time. We also didn’t have time to take it further.
That being said, we’re open to further improve it. However, image manipulation in the SketchUp API is somewhat of a stretch. In the C API you can easily enough grab a C/C++ library suited to your need.
But I do understand the gap it leaves Ruby. If we were to add something in this respect, then we need a set of clear real world examples of what people wish to do. High level first, then we can explore what that means for the low level.
I understand what you’re saying from a perspective of wanting to manipulate image data. But the intent was to make it easier to read the data - which many extensions already did by means of writing to disk only to read it back in again.
I really encourage you to add a feature request in the issue tracker with a concrete set of things you’d like to do with an image manipulating API. The scope for that is currently all to wide open - we need feedback from developers to narrow down to a set of functionality that fits into a generic API.
Seeing how you seem to be comfortable with Ruby’s speed so far. Would it be of interest if we started a repository with example image manipulation in Ruby? Something everyone could contribute to? That would address some of the wide scope of this. If we had a generic wide reference people contributed to, us included, then developers can pull in what they need.
That would also fit better with the core design of API’s keeping in lean. And would act as a substitute for a standard SU library (until we can work something out in that respect.)
I thought ImageRep deals with OS Endian -ness transparently…
these yield exactly the same result on a mac…
r, g, b, a = substitute_proc.call(*pixel)
new_data << r.chr << g.chr << b.chr
r, g, b, a = substitute_proc.call(*pixel.reverse)
new_data << b.chr << g.chr << r.chr
you have a double reverse in the windows version…
john
I still think that for new domain of the API, it is useful to ask a panel of developers, so that you can measure the implications and then make your own decision in consideration of the Sketchup resource plan
I think the view#write_imagerep is still something desirable. Maybe it is available in the C API, but as far as I know, it is not yet in the Ruby API.
Anyway, my railing against the ImageRep API was temporary, primarily because of the lack of any documentation and examples. I have now understood how to deal with it. Still, there remains the question of performance, and high-level manipulation methods written in C under the cover would beat by far any calculation loop performed in Ruby. 70 ms on a 256x256 image may mean several seconds on an image big as the viewport!
I support the idea of a place where practices and code snippets can be shared for developers. In the present forum, developers contributions get buried with time, and anyway obfuscated within the thread discussion. The idea of this dedicated repository would be to centralize tip and tricks, as well as examples of code for the existing API.
Well, I could check that for Windows the order of bytes your read from the data are (B, G, R), so reverse order. For Mac, I assumed that you read bytes in the correct order (R, G, B), based on Thomthom’ advice. I don’t have a Mac, so I can’t check. If you have one, please confirm.
I tested on this image…
and ran a modified version of your code to output two files, one using the mac code, the other the windows code…
I don’t think you need the two versions with imageRep, that’s only if using pack/unpack directly on external files…
john
Do you mean that when you slice a pixel in, say 3 bytes, the order is [B, G, R] on both Mac and Windows.
I was making reference to a post from thomthom, earlier in this thread, where he exposed the following code
22. # From C API documentation on SUColorOrder
23. #
24. # > SketchUpAPI expects the channels to be in different orders on
25. # > Windows vs. Mac OS. Bitmap data is exposed in BGRA and RGBA byte
26. # > orders on Windows and Mac OS, respectively.
27. def self.color_to_32bit(color)
28. r, g, b, a = color.to_a
29. IS_WIN ? [b, g, r, a] : [r, g, b, a]
30. endTo be clearer, what I am saying is that, on Windows, the pixel is encoded as [B, G, R]
imgrep.data.bytes.each_slice(imgrep.bits_per_pixel / 8) do |pixel|
b, g, r, a = pixel
....
end if you only have R pixels, G pixels, B pixels, reversing them once should flip the image…
if TT’s code is needed for ImageRep on windows, a single reverse would be required…
As your code has a double flip, if the resulting images are correct, than that implies the TT code isn’t needed…
sorry if I not explaining myself clearly…
see what happens with the example file on a PC…
john
I reverse it, because I want to pass [R, G, B] to my substitution proc, regardless of the platform, This substitution proc returns also [R, G, B].
Then, to put it back into the new data, I need to push them in reverse order [B, G, R].
Hence what you see as a ‘double’ reverse.
Here is the more detailed code
imgrep.data.bytes.each_slice(imgrep.bits_per_pixel / 8) do |pixel|
b, g, r, a = pixel # read in reverse order
r, g, b, a = substitution_proc.call(r, g, b, a) # normal order [r, g, b] input and output
new_data << b.chr << g.chr << r.chr # push back in reverse order
end QUESTION: does it work the same on Mac?