{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "# Tutorial01 - Getting Started\n",
                "\n",
                "## Overview\n",
                "\n",
                "🚀 This tutorial demonstrates a minimal application of XRFeitoria. By the end of this tutorial, you will be able to:\n",
                "\n",
                "- Open Blender/Unreal Engine by XRFeitoria\n",
                "- Import a mesh\n",
                "- Add a camera\n",
                "- Render images and get annotations of the mesh"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 1. Import XRFeitoria\n",
                "\n",
                "After installing ``pip install xrfeitoria[vis]``, you can import it as follows:"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import xrfeitoria as xf"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 2. Choose engine\n",
                "\n",
                "``XRFeitoria`` supports both ``Blender`` and ``Unreal Engine``. Choose your engine and replace the following ``engine_exec_path`` with your own engine path.\n",
                "\n",
                "Then, initialize XRFeitoria and open the engine by ``xf.init_blender`` or by ``xf.init_unreal``."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Replace with your executable path\n",
                "engine_exec_path = 'C:/Program Files/Blender Foundation/Blender 3.3/blender.exe'\n",
                "# engine_exec_path = 'C:/Program Files/Epic Games/UE_5.2/Engine/Binaries/Win64/UnrealEditor-Cmd.exe'"
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "from pathlib import Path\n",
                "\n",
                "exec_path_stem = Path(engine_exec_path).stem.lower()\n",
                "if 'blender' in exec_path_stem:\n",
                "    # Open Blender\n",
                "    render_engine = 'blender'\n",
                "    xf_runner = xf.init_blender(exec_path=engine_exec_path, background=False, new_process=True)\n",
                "elif 'unreal' in exec_path_stem:\n",
                "    # Unreal Engine requires a project to be opened\n",
                "    # Here we use a sample project, which is downloaded from the following link\n",
                "    # You can also use your own project\n",
                "    import shutil\n",
                "\n",
                "    from xrfeitoria.utils.downloader import download\n",
                "    unreal_project_zip = download(url='https://github.com/openxrlab/xrfeitoria/releases/download/v0.1.0/UE--XRFeitoriaUnreal_Sample.zip', \n",
                "                                    dst_dir=\"./tutorial01/assets/\")\n",
                "    shutil.unpack_archive(filename=unreal_project_zip, extract_dir='./tutorial01/assets/')\n",
                "\n",
                "    # Open Unreal Engine\n",
                "    render_engine = 'unreal'\n",
                "    xf_runner = xf.init_unreal(exec_path=engine_exec_path, \n",
                "                                background=False, \n",
                "                                new_process=True, \n",
                "                                project_path='./tutorial01/assets/UE_sample/UE_sample.uproject')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "✨ Now you can see a new Blender/Unreal Engine process has been started.\n",
                "\n",
                "- Blender\n",
                "![](https://github.com/user-attachments/assets/e02ea52f-658f-4031-8439-2d5c3ade70ae)\n",
                "\n",
                "- Unreal Engine\n",
                "![](https://github.com/user-attachments/assets/d93fee50-7117-4503-b901-40c154fd1e7e)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 3. Import a mesh\n",
                "\n",
                "Download the scanned Koupen Chan model by the following cell."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "from xrfeitoria.utils.downloader import download\n",
                "\n",
                "# Download the Koupen-chan model\n",
                "kc_path = download(url='https://github.com/openxrlab/xrfeitoria/releases/download/v0.1.0/assets--koupen_chan.fbx', \n",
                "                   dst_dir=\"./tutorial01/assets/\")"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Import the .fbx file to create an ``Actor`` instance.\n",
                "\n",
                "``Actor`` is the container of a mesh. By using ``Actor``, you can place a mesh in the space and set its transform data (location, rotation and scale)."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# Import the Koupen-chan\n",
                "actor_kc = xf_runner.Actor.import_from_file(file_path=kc_path, stencil_value=255)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Switch to the engine window, and you can see the Koupen chan has been *imported*. The space to place ``Actor`` is called ``Level``, and you can *add*, *remove*, or *modify* ``Actor`` in the `Level`.\n",
                "\n",
                "- Blender\n",
                "![](https://github.com/user-attachments/assets/f38220c6-6f72-4d54-a304-9b91e0739aa5)\n",
                "\n",
                "- Unreal Engine\n",
                "![](https://github.com/user-attachments/assets/8dbcfd99-8fb0-41da-a4ff-3f7aa101375b)"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "If you use ``Unreal Engine``, the ``Level`` should be saved after been modified."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "# save the level\n",
                "if render_engine == 'unreal':\n",
                "    xf_runner.utils.save_current_level()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 4. Add a sequence\n",
                "\n",
                "``Sequence`` is a multifunctional class in XRFeitoria. It can be used for:\n",
                "\n",
                "- rendering\n",
                "- adding transform keys\n",
                "- grouping different objects\n",
                "\n",
                "Here, we use it for rendering. Firstly we will add a ``Camera`` in the ``Sequence`` by using the function `spawn_camera` and set its location, rotation, and focal length. Then, we will add a render job to the renderer and set the render settings by using the function ``add_to_renderer``."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "from xrfeitoria.data_structure.models import RenderPass\n",
                "\n",
                "# Use `with` statement to create a sequence, and it will be automatically close the sequence after the code block is executed.\n",
                "sequence_name = 'MySequence'\n",
                "with xf_runner.sequence(seq_name=sequence_name, replace=True) as seq:\n",
                "\n",
                "    # Add a camera and make it look at the koupen-chan\n",
                "    camera_location = (0.0, -0.8, 0.0)\n",
                "    camera_rotation = xf_runner.utils.get_rotation_to_look_at(location=camera_location, target=actor_kc.location)\n",
                "    camera = seq.spawn_camera(location=camera_location, rotation=camera_rotation, fov=90)\n",
                "\n",
                "    # Add a render job to renderer\n",
                "    # In render job, you can specify the output path, resolution, render passes, etc.\n",
                "    # The ``output_path`` is the path to save the rendered data.\n",
                "    # The ``resolution`` is the resolution of the rendered image.\n",
                "    # The ``render_passes`` define what kind of data you want to render, such as img, mask, normal, etc.\n",
                "    # and what kind of format you want to save, such as png, exr, etc.\n",
                "    seq.add_to_renderer(\n",
                "        output_path=f'./tutorial01/outputs/{render_engine}/',\n",
                "        resolution=(1280, 720),\n",
                "        render_passes=[RenderPass('img', 'png'),\n",
                "                       RenderPass('mask', 'exr'),\n",
                "                       RenderPass('normal', 'exr'),\n",
                "                       RenderPass('diffuse', 'exr')]\n",
                "    )"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 5. Render\n",
                "\n",
                "Use the following cell to render and save the images to the ``output_path`` you set in ``seq.add_to_renderer`` above."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "xf_runner.render()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Check the ``output_path``, and you can see the rendered images and various annotations. The following code shows the rendered image and the corresponding annotations."
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "import matplotlib.pyplot as plt\n",
                "\n",
                "from xrfeitoria.utils.viewer import Viewer\n",
                "\n",
                "xf_viewer = Viewer(sequence_dir=f'./tutorial01/outputs/{render_engine}/{sequence_name}/')\n",
                "img = xf_viewer.get_img(camera_name=camera.name, frame=0)\n",
                "diffuse = xf_viewer.get_diffuse(camera_name=camera.name, frame=0)\n",
                "mask = xf_viewer.get_mask(camera_name=camera.name, frame=0)\n",
                "normal = xf_viewer.get_normal(camera_name=camera.name, frame=0)\n",
                "\n",
                "plt.figure(figsize=(20, 20))\n",
                "\n",
                "plt.subplot(1, 4, 1)\n",
                "plt.imshow(img)\n",
                "plt.axis('off')\n",
                "plt.title('img')\n",
                "\n",
                "plt.subplot(1, 4, 2)\n",
                "plt.imshow(diffuse)\n",
                "plt.axis('off')\n",
                "plt.title('diffuse')\n",
                "\n",
                "plt.subplot(1, 4, 3)\n",
                "plt.imshow(mask)\n",
                "plt.axis('off')\n",
                "plt.title('mask')\n",
                "\n",
                "plt.subplot(1, 4, 4)\n",
                "plt.imshow(normal)\n",
                "plt.axis('off')\n",
                "plt.title('normal')"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "> Hint: When using Unreal Engine, if the image of the mask looks weird, try running the notebook again."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 6. Final step\n",
                "\n",
                "🥳 This is a good start! Finally, **Do remember** to close the engine. "
            ]
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": [
                "xf_runner.close()"
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "Ref to [api docs](https://xrfeitoria.readthedocs.io/en/latest/apis/xrfeitoria.html), you can always use ``with`` statement to ensure the engine is closed when the codes are finished."
            ]
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": [
                "## 7. Conclusion\n",
                "\n",
                "In this tutorial, we imported a mesh and rendered images for it. To accomplish this procedure, essential steps should be taken:\n",
                "\n",
                "- Initialization\n",
                "- Import an `Actor`\n",
                "- Add a `Sequence`\n",
                "- Add a `Camera`\n",
                "- Render\n",
                "\n",
                "It is worth mention that while ``Sequence`` does not directly perform rendering, the creation of ``Sequence`` is necessary for adding cameras and submitting rendering jobs. And the detailed definitions of the classes [Actor](https://xrfeitoria.readthedocs.io/en/latest/apis/actor.html), [Camera](https://xrfeitoria.readthedocs.io/en/latest/apis/camera.html), and [Sequence](https://xrfeitoria.readthedocs.io/en/latest/apis/sequence.html) can be referred to the documentation."
            ]
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "blender-pipeline",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.10.12"
        },
        "orig_nbformat": 4
    },
    "nbformat": 4,
    "nbformat_minor": 2
}