Segmentation Data Background

During the vizHOME project, objects were manually segmented for purposes of running different portions of the study. As a potential sample use case, this information could be used to test against automatic segmentation algorithms to determine "correctness" of the automatic segmentation. The types of objects that were segmented include:

Bag
Basket
Bathtub
Bed
Books
Bookshelf
Cabinets
Calendar
Cane
Chair
Clock
Clock
Closet
Closet
Coffee Marker
Coffee Table
Computer
Couch
Counters
Desk
Dining Table
Door
Door
Drawers
Dresser
Dresser Top
End Table
File Cabinet
Laptop
Lightswitch
Magazine Rack
Med cabinet
Microwave
Mirror
Nightstand
Picture Frame
Pill Bottles
Printer
Refrigerator
Refrigerator artifacts
Shelves
Sink
Smartphone
Sofa
Stairs
Storage box
Stove
TV
TV Remote
TV Tray
Telephone
Toilet
Wastebasket

Segmentation Data Format

The segmentation data is provided in a text file, where each line is a separate object within the home that has been segmented.

Each line of segmentation data is provided in the following format:

<name of object><single space><16 floating point numbers separated by 2 spaces>

The 16 floating point numbers represent a transformation that defines an oriented bounding box that surrounds a single object in the home. The identity matrix would represent a 1 meter x 1 meter x 1 meter box whose center point is 0,0,0 in 3D space.

Sample code for Parsing Segmentation Data

The following code provides an example in C# Unity3D for parsing and creating bounding boxes for each object in the segmentation data under a parent Unity game object:

using System.Collections;
using System.Collections.Generic;
using System.IO;
using System;
using UnityEngine;

public class LoadBoundingBoxes : MonoBehaviour {

    public string fileName;
    public GameObject boundingBoxPrefab;

	// Use this for initialization
	void Start () {
		
	}
	
	// Update is called once per frame
	void Update () {
		
	}

    [ContextMenu("LoadFile")]
    void LoadData()
    {
        if (fileName.Length > 0)
        {
            StreamReader reader = new StreamReader(fileName);
            string line;
            while ((line = reader.ReadLine()) != null)
            {
                string name;
                string[] ss = line.Split(
                        new[] { " " }, 
                        StringSplitOptions.RemoveEmptyEntries
                    );
                //Debug.Log(ss.Length);
                name = ss[0];
                Vector3 pos = new Vector3(
                         (float)double.Parse(ss[13]), 
                         (float)double.Parse(ss[15]), 
                        (float)-double.Parse(ss[14])
                    );
                GameObject bounds = Instantiate(
                        boundingBoxPrefab, 
                        pos, 
                        UnityEngine.Quaternion.identity, 
                        gameObject.transform
                    );
                Vector3 xScale = new Vector3(
                        (float)double.Parse(ss[1]), 
                        (float)double.Parse(ss[2]), 
                        (float)double.Parse(ss[3])
                    );
                Vector3 zScale = new Vector3(
                        (float)double.Parse(ss[5]), 
                        (float)double.Parse(ss[6]), 
                        (float)double.Parse(ss[7])
                    );
                Vector3 yScale = new Vector3(
                        (float)double.Parse(ss[9]), 
                        (float)double.Parse(ss[10]), 
                        (float)double.Parse(ss[11])
                    );
                bounds.transform.localScale = new Vector3(
                        xScale.magnitude, 
                        zScale.magnitude, 
                        yScale.magnitude
                    );

                Vector3 xA = xScale.normalized;
				Vector3 yA = yScale.normalized;
				Vector3 zA = zScale.normalized;
                Vector4 xAxis = new Vector4(xA.x, yA.x, zA.x, 0f);
                Vector4 yAxis = new Vector4(xA.y, yA.y, zA.y, 0f);
                Vector4 zAxis = new Vector4(xA.z, yA.z, zA.z, 0f);
                Vector4 wAxis = new Vector4(1f, 1f, 1f, 0f);
                Matrix4x4 m = new Matrix4x4(xAxis, yAxis, zAxis, wAxis);
                bounds.transform.rotation = m.rotation;
                bounds.name = name;
            }
        }
    }       
}