Determine texture color of a face

Hello again,

I set out to try to create a ruby script that could select faces by their average color, but I got hung up right away in trying to understand the whole UVQ to XYZ texture mapping thing.

I read several forum topics and tried various ideas, however, I got kinda stuck as well as confused.

I’m attaching a model which I found in the 3D warehouse. It isn’t my model and I hope it’s ok to use it for this example.

Here is the example model:
select_by_color_example.skp (5.5 MB)

Here I’ve selected one face in the skintone area and as you can see, the correct RGB value is highlighted [254,188,155]
rgb_skin_tone

I found a pure ruby gem called ChunkyPNG which can manipulate PNG images and I was able to install into sketchup easily

Gem.install "chunky_png"

Here is my script so far. In it I select a face in a textured model and then obtain the UVQ coordinates of the face vertices and then try to map them to the texture png itself.

The output of the script gives quite incorrect colors, somewhere in the dark green/orange/brown area.

Thank you for any and all clues or hints
-j_jones

require 'chunky_png'

module JJones
  def self.getUVs(front, faceEnt, tw)
    verts = faceEnt.outer_loop.vertices
    points = verts.collect { |v| v.position }
    # uv helper allows you to determine the location (UV coordinates) of a texture on a face
    uvh = faceEnt.get_UVHelper(true, true, tw)
    if front
      uvs = points.collect {|p| uvh.get_front_UVQ(p)}
    else
      uvs = points.collect {|p| uvh.get_back_UVQ(p)}
    end
    
    for i in (0..uvs.length-1)
      uvs[i] = uvs[i].to_a # convert point3d to array
      uv_w = uvs[i].z
      uvs[i].x = uvs[i].x / uv_w
      uvs[i].y = uvs[i].y / uv_w
      uvs[i].y = uvs[i].y / uv_w
    end
    return uvs
  end # getUVs

  def self.describe_face(face, tw=Sketchup.create_texture_writer)
    texture = face.material.texture
    height = texture.height
    width = texture.width
    image_height = texture.image_height
    image_width = texture.image_width
    avg_color = texture.average_color
    texture_file_name = texture.filename
    image = get_texture_png(texture_file_name)

    back_uvs = getUVs(false, face, tw)
    front_uvs = getUVs(true, face, tw)
    return [texture_file_name, texture, image, image_height, image_width, avg_color, back_uvs, front_uvs]
  end # describe_face
  
  def self.get_texture_png(texture_file_name)
    image = ChunkyPNG::Image.from_file(texture_file_name)
    return image
  end # get_texture_png

  def self.get_pixel_color_from_uv(png_image, u, v)
    texture_width = png_image.width
    texture_height = png_image.height
    
    pixel_x = (u * texture_width).to_i
    pixel_y = (v * texture_height).to_i
    if (pixel_x >= texture_width) || (pixel_y >= texture_height)
      # print("pixel(s) #{pixel_x}, #{pixel_y} out of bounds for image?")
      return nil
    end

    # sample the color at these pixel coords
    pixel_color = png_image[pixel_x, pixel_y]
    r = ChunkyPNG::Color.r(pixel_color)
    g = ChunkyPNG::Color.g(pixel_color)
    b = ChunkyPNG::Color.b(pixel_color)
    a = ChunkyPNG::Color.a(pixel_color)
    return [r,g,b,a]
  end # get_pixel_color_from_uv

  def self.find_similar_faces()
    model = Sketchup.active_model
    entz = model.active_entities
    selection = model.selection
    faces = selection.grep(Sketchup::Face)
    return unless faces.any?

    face = faces[0] # just use the first selected face for this example
    texture_file_name, texture, image_png, image_height, image_width, avg_color, back_uvs,         front_uvs = JJones.describe_face(face)

    print("FRONT_UVS: #{front_uvs}")
  
    front_uvs.each do |p|
      u,v = p
      pixel_color = JJones.get_pixel_color_from_uv(image_png, u, v)
      if ! pixel_color.nil?
        print("pixel color: #{pixel_color}")
      end
    end

    print("BACK_UVS: #{back_uvs}")
    back_uvs.each do |p|
      u,v = p
      pixel_color = JJones.get_pixel_color_from_uv(image_png, u, v)
  i    f ! pixel_color.nil?
        print("pixel color: #{pixel_color}")
      end
    end  
  end
end # module JJones

# first select a face, then run this to find faces of a similar color, doesn't work yet
JJones.find_similar_faces()

BTW, here is the texture associated with the model. Again, not my model, hope it’s ok to use this for an example.

You cannot use the API’s ImageRep class?

I cannot because I’m using Sketchup 2017 and that feature came out in 2018

Well, I went to the crazy extreme of mapping out the pixels of the texture into the 3d space via little colored cubes…

And now I see the problem is that the way I’m traversing the image, it is flipped along the y axis…

Now I maybe can fix it

Finally got it going. I had to flip the image along the y-axis and then the uvs matched up.

Not sure if it will work for all cases, but I’m happy enough with this example code for now.

select_by_texture_color

Here’s the script. You select one face, it calculates the average color of the face and then greps for all faces of a similar color within a specified tolerance.

require 'chunky_png'

module JJones

  def self.getUVs(front, faceEnt, tw)
    verts = faceEnt.outer_loop.vertices
    points = verts.collect { |v| v.position }
    # uv helper allows you to determine the location (UV coordinates) of a texture on a face
    uvh = faceEnt.get_UVHelper(true, true, tw)
    if front
      uvs = points.collect {|p| uvh.get_front_UVQ(p)}
    else
      uvs = points.collect {|p| uvh.get_back_UVQ(p)}
    end
    
    for i in (0..uvs.length-1)
      uvs[i] = uvs[i].to_a # convert point3d to array
      uv_w = uvs[i].z
      uvs[i].x = uvs[i].x / uv_w
      uvs[i].y = uvs[i].y / uv_w
      uvs[i].y = uvs[i].y / uv_w
    end
    return uvs
  end # getUVs

  def self.describe_face(face, tw=Sketchup.create_texture_writer)
    texture = face.material.texture
    if ! texture.nil?
      height = texture.height
      width = texture.width
      image_height = texture.image_height
      image_width = texture.image_width
      texture_file_name = texture.filename
      image = get_texture_png(texture_file_name)
    end

    # back_uvs = getUVs(false, face, tw)
    # front_uvs = getUVs(true, face, tw)
    return [texture_file_name, texture, image, image_height, image_width]
  end # describe_face
  
  def self.swap_image_y_direction(image_png)
    # create a new blank image
    # copy the pixels over to the new image
    # but swap the y direction
    width = image_png.width
    height = image_png .height
    new_image = ChunkyPNG::Image.new(width, height)
    new_y = 0
    last_row = height-1
    last_row.step(0, -1) do |y|
      image_png.row(y).each_with_index do |pixel,x|
        pixel_color = image_png[x, y]
        new_image[x,new_y] = pixel_color
      end
      new_y += 1
    end
    new_image
  end
  
  def self.get_texture_png(texture_file_name)
    image = ChunkyPNG::Image.from_file(texture_file_name)
    new_image = self.swap_image_y_direction(image)
    return new_image
  end # get_texture_png
  
  def self.get_pixel_color_from_uv(png_image, u, v)
    texture_width = png_image.width
    texture_height = png_image.height
    
    pixel_x = (u * texture_width).to_i
    pixel_y = (v * texture_height).to_i
    if (pixel_x >= texture_width) || (pixel_y >= texture_height)
      # print("pixel(s) #{pixel_x}, #{pixel_y} out of bounds for image?")
      return nil
    end

    # sample the color at these pixel coords
    pixel_color = png_image[pixel_x, pixel_y]
    r = ChunkyPNG::Color.r(pixel_color)
    g = ChunkyPNG::Color.g(pixel_color)
    b = ChunkyPNG::Color.b(pixel_color)
    a = ChunkyPNG::Color.a(pixel_color)
    return [r,g,b,a]
  end # get_pixel_color_from_uv
  
  def self.get_average_color(colors)
    num_pixels = colors.length
    if num_pixels == 0
      return nil
    end
    total_color = { r: 0, g: 0, b: 0, a: 0 }
    
     colors.each do |color|
       total_color[:r] += color[0]
       total_color[:g] += color[1]
       total_color[:b] += color[2]
       total_color[:a] += color[3]
    end
    
    avg_color = {
      r: total_color[:r] / num_pixels,
      g: total_color[:g] / num_pixels,
      b: total_color[:b] / num_pixels,
      a: total_color[:a] / num_pixels 
    }
    
    result = [avg_color[:r],avg_color[:g],avg_color[:b],avg_color[:a]]
    #print("average color: #{avg_color}, result: #{result}")
    return result
  end
  
  def self.get_avg_face_color(image, face, front, tw)
    #print("get_avg_face_color")
    # compute the average color based on corner points of face
    colors = []
    uvs = JJones.getUVs(front, face, tw)
    uvs.each do |p|
      u,v = p
      pixel_color = JJones.get_pixel_color_from_uv(image, u, v)
      if ! pixel_color.nil?
        colors << pixel_color
      end
    end
    avg_color = JJones.get_average_color(colors)
    #print("get_avg_face_color: avg_color: #{avg_color}")
    return avg_color
  end
  
  def self.similar_color?(color1, color2, tolerance = 10)
    (color1[0] - color2[0]).abs <= tolerance &&
    (color1[1] - color2[1]).abs <= tolerance &&
    (color1[2] - color2[2]).abs <= tolerance &&
    (color1[3] - color2[3]).abs <= tolerance
  end
  
  def self.find_similar_face_colors(tolerance)
    model = Sketchup.active_model
    entz = model.active_entities
    selection = model.selection
    faces = selection.grep(Sketchup::Face)
    return unless faces.any?
    
    face = faces[0] # use the first face to determine the texture info
    texture_file_name, texture, image, height, width = JJones.describe_face(face)
    print("texture_file_name: #{texture_file_name}")
    print("image height: #{height}, width: #{width}")
    
    tw=Sketchup.create_texture_writer
    
    # get color of target face
    target_color = JJones.get_avg_face_color(image, face, true, tw)
    print("target color: #{target_color}")
    
    # now grep all faces and collect those that are similar color
    faces = entz.grep(Sketchup::Face)
    face_colors = {}
    similar_colored_faces = []
    
    faces.each_with_index do |face,i|
      face_color = JJones.get_avg_face_color(image, face, true, tw)
      if face_color.nil?
        next
      end
      if JJones.similar_color?(target_color, face_color, tolerance)
        similar_colored_faces << face
      end
    end
    
    model.selection.add(similar_colored_faces)
  end
  
end # module JJones

# first select a face of the desired color, then run this method
tolerance = 80
JJones.find_similar_face_colors(tolerance)

There is one weird side effect I’m not sure why it’s happening, but something is drawing construction points further out in the model space. I have no clue why that would happen or where in the code it would happen, but if anyone knows, please advise.

Thank you
-j_jones

3 Likes

There is was a bunch of code following your JJones module that should be within a method of the module, not executing at the top level.

cleaned up and reposted, hope that’s better.

1 Like