Move all the point and color data into the SceneObjects and populate all the different vbos by looping over all the SceneObjects.