CARLA Simulator: Difference between revisions

 
(2 intermediate revisions by the same user not shown)
Line 35: Line 35:
** You will need an Epic Games account linked to your GitHub to have access to the source code.
** You will need an Epic Games account linked to your GitHub to have access to the source code.
* Clone the carla repo and download assets.
* Clone the carla repo and download assets.
<pre>
# Launch Unreal Engine
make launch
# Build Carla
make package
</pre>
}}
}}


Line 41: Line 50:


===RGB Image===
===RGB Image===
;Disabling Effects for Stitching
See [https://github.com/carla-simulator/carla/issues/2185  Disable all lens simulating effects on cameras ] 
<pre>
# carla/CarlaUE4/Config/DefaultEngine.ini
# Add to section [/Script/Engine.RendererSettings]
r.BlackBorders=0
r.DepthOfFieldQuality=0
r.DisableDistortion=1
r.MotionBlurQuality=0
r.SceneColorFringeQuality=0 # might be chromatic aberrations
r.Tonemapper.Quality=0 # might be vignette
</pre>
===Depth Image===
===Depth Image===
Depths images encode as z-depth to camera as 8-bit images.   
Depths images encode as z-depth to camera as 8-bit images.   
Line 55: Line 78:


{{hidden | Z-depth to Euclidean Depth |
{{hidden | Z-depth to Euclidean Depth |
 
See [https://github.com/carla-simulator/carla/issues/2287 https://github.com/carla-simulator/carla/issues/2287] for a formula to convert z-depth to euclidean depth.
Below is code to convert z-depth to euclidean depth for an equirectangular image stitched from cubemaps. 
Adapt it accordingly.
<syntaxhighlight lang="python">
theta, phi = np.meshgrid((np.arange(width) + 0.5) * (2 * np.pi / width),
                        (np.arange(height) + 0.5) * (np.pi / height))
uvs, uv_sides = my_helpers.spherical_to_cubemap(theta.reshape(-1),
                                              phi.reshape(-1))
cubemap_uvs = uvs.reshape(height, width, 2)
# cubemap_uv_sides = uv_sides.reshape(height, width)
world_coords = my_helpers.spherical_to_cartesian(
  cubemap_uvs[:, :, 0] * np.pi / 2 + np.pi / 4,
  cubemap_uvs[:, :, 1] * np.pi / 2 + np.pi / 4)
cos_angle = world_coords @ np.array([0, 0, 1], dtype=np.float32)
return depth_image / cos_angle[np.newaxis, :, :, np.newaxis]
</syntaxhighlight>
 
* Here <code>cubemap_uvs</code> are the UVs of each equirectangular pixel within the cubemap.
 
Spherical to cubemap is defined as follows:
<syntaxhighlight lang="python">
def spherical_to_cubemap(theta, phi):
  """Converts spherical coordinates to cubemap coordinates.
 
  Args:
    theta: Longitude (azimuthal angle) in radians. [0, 2pi]
    phi: Latitude (altitude angle) in radians. [0, pi]
 
  Returns:
    uv: UVS in channel_last format
    idx: Side of the cubemap
 
  """
 
  u = np.zeros(theta.shape, dtype=np.float32)
  v = np.zeros(theta.shape, dtype=np.float32)
  side = np.zeros(theta.shape, dtype=np.float32)
  side[:] = -1
 
  for i in range(0, 4):
    indices = np.logical_or(
        np.logical_and(theta >= i * np.pi / 2 - np.pi / 4, theta <=
                      (i + 1) * np.pi / 2 - np.pi / 4),
        np.logical_and(theta >= i * np.pi / 2 - np.pi / 4 + 2 * np.pi, theta <=
                      (i + 1) * np.pi / 2 - np.pi / 4 + 2 * np.pi))
    u[indices] = np.tan(theta[indices] - i * np.pi / 2)
    v[indices] = 1 / (
        np.tan(phi[indices]) * np.cos(theta[indices] - i * np.pi / 2))
    side[indices] = i + 1
  top_indices = np.logical_or(phi < np.pi / 4, v >= 1)
  u[top_indices] = -np.tan(phi[top_indices]) * np.sin(theta[top_indices] -
                                                      np.pi)
  v[top_indices] = np.tan(phi[top_indices]) * np.cos(theta[top_indices] - np.pi)
  side[top_indices] = 0
  bottom_indices = np.logical_or(phi >= 3 * np.pi / 4, v <= -1)
  u[bottom_indices] = -np.tan(phi[bottom_indices]) * np.sin(
      theta[bottom_indices])
  v[bottom_indices] = -np.tan(phi[bottom_indices]) * np.cos(
      theta[bottom_indices])
  side[bottom_indices] = 5
 
  assert not np.any(side < 0), "Side less than 0"
 
  return np.stack(((u + 1) / 2, (-v + 1) / 2), axis=-1), side
</syntaxhighlight>
}}
}}