Innovations Report 2003

"Towards an expressive and subversive projection"

ben toogood

document tested in iexplore5.0 and is best viewed at text size medium
joiner banner

Contents

Abstract
This report documents the research into and experimentation with projection as creative device. It assesses the visual consequences of such a technique through the analysis of an animation produced in the style.

1 Aims and Objectives

2 Introduction

3 The case for a new perspective
3.1 Ideas of perspective
       3.1.1 Projection and its use
          3.1.2 Mathematical projection
          3.1.3 Painters projection
       3.1.4 Perspective, its use, and limitations
3.2 Ideas of seeing and representing
       3.2.1 Children’s symbolic sense of space
       3.2.2 Incompleteness of image
       3.2.3 Continuity editing
       3.2.4 Hockney’s joiner works
3.3 Issues relating to computer animation
       3.3.2 Work previously done in field of expanded perspective
4 Tool Development
4.1 Objectives for the tool
       4.1.1 Automation requirements
       4.1.2 Need for absolute control
4.2 Scripted approach
       4.2.1 Separation of tasks
4.3 Camera layout and control
       4.3.1 Making use of Maya’s tools to provide an environment for camera design
4.4 Rendering and tile assembly

5 Exploration of consequences
5.1 Tests run parallel with tool development
5.2 Production of animated sequence
       5.2.1 Theme of animation in relation to theme of project
       5.2.2 Objectives for animation
       5.2.3 Production limitations

6 Evaluation
6.1 Suitability for narrative, expression, or abstraction?
6.2 Are the images compelling or confusing?
6.3 Project assessment
6.4 Specific futher developements

7 Bibliography and Acknowledgements
7.1 Bibliography
7.2 Acknowledgements

8 Appendices
8.1 Hockney joiner images
8.2 Tool code files
8.3 Complete examples from development stage
8.4 Reference papers

joiner banner

___________________________________________________


contents

1 Aims and Objectives

Aims
Explore the representation of a three-dimensional scene from multiple simultaneous viewpoints.
Develop a tool set to aid this.
Assess the visual consequences and possibilities of this concept/technique.
Objectives
Develop an understanding of projection and perspective representation in relation to painting and animation.
Produce a tool set integrated with the systems 'Maya' and 'Shake' to allow the rendering and assembly of multiple viewpoint image sequences.
Execute a short animation exploring any unique aesthetic and temporal properties possible using this device.
Assess the effectiveness of the resultant animation sequences in terms of spatial and atmospheric depiction and suitability for narrative or abstract animation production.

___________________________________________________


contents

2 Introduction


specialist project  ben toogood 2002
Fig 2.1 Specialist Project

The image left is taken from a project completed in 2002, which used projected shadows to introduce ambiguity into rendered images of three dimensional scenes. By not viewing the objects in the scene directly, a less explicit description is presented to the viewer. The new implied shape with its indistinction hopefully allows greater interpretation to take place, and situates more of the animation in the imagination of the observer. It was intended that this technique would generate a richer experience of the scenario and engage a viewer who could have previously been more passive.

The resultant optical illusions of the cast shadow technique proved visually and intellectually involving, but in a way that fought against the depiction of spatial relationships and volumes. It was found that simple geometries in simple arrangements were most successful, unlike the complex and detailed subjects normally dealt with in 3D computer animation. The project had been born out of a desire to subvert the representation of form and space normally permitted in 3D animation systems which was achieved, however in a limited manner.

This report details the investigation into the possibilities and potential of another technique for the perversion of three-dimensional space in rendered CGI. The restrictions of the viewing model employed in most animation systems, that being a perspective pin hole based camera, are still felt by the author to be in conflict with a disire to produce expressive and emotive animation with computers. Recently simulations of traditional media through the use of computer animation have recieved attention both in research and experimental productions. Often, a wish to produce more subjective rather than scientific images is cited as a motivation for 'Non photorealistic' rendering (NPR). These works are generally concerned with the mark making process, rather than the viewing process, and often function as post processess on rendered images. I belive that an ability to wilfully and arbitarily modify the proceedure for mapping the three-dimensional scene onto the two-dimensional viewing medium will add to the computer artists ability to represent the/a world as they percieve it. The painter and artist David Hockey says of traditonal stills photography:

"Photography is alright if you don't mind looking at the world from the point of view of a paralysed cyclops, for a split second." (Hockney)

We will examine over the course of the remainder of this report the possibility of looking at our (computer)world from many points of view, and we will consider the consequences of this expanded viewing device in the time based medium of animation. The next section will deal with the conceptual issues behind viewing which need to be considered in the exploratory process, and will make a case for the proposed system for generating interpretive projections. Then the attempt at implementation will be reviewed, followed by an examination of the aesthetic consequences revealed through the production of a short animated sequence. An assement of the technique and its relation with animation will close the report.

___________________________________________________


contents

3 The case for a new perspective

In order to argue for the new technique proposed in this project it is necessary to review various concepts and discuss the faults and benefits of specific issues. Hopefully a progression will form, and by the end of this section we will have a more complete understanding of viewing in 3D animation and of how we could expand and improve it. To begin, we will analyse projection and specifically the projection most usual in computer animation, perspective.

3.1 Ideas of perspective
       3.1.1 Projection and its use

Projection forms the basis of all viewing. It is the representation of three-dimensional space in two-dimensions. It is common to optic vision, photography, painting, design, maths, and (combining all of the previous) computer animation. The rule set or cunning subterfuge employed to describe the three dimensions in two is named the projection. To form a model for viewing and projection let us consider three cases. In each we must have a viewing device and a viewing surface, both described by Upstill(1989 pg. 4)

"The purpose of the viewing device is to organize, or project, the light as a coherent image onto the viewing surface"

Now our cases:
   viewing device   viewing surface
       human eye       visual cortex
       painter       canvas
       virtual camera       pixmap image array

Upstill is discussing viewing devices in the sense of optics in order to present the system use by the Renderman specification. To him the issue of coherency is essential as Renderman was designed for use in photo-realistic rendering and the simulation of conventional filming. For these purposes it is desirable for the arrangement of virtual light on the viewing surface to be coherent and predictable, analogous to the arrangement occurring in camera systems. Are there other forms that projection can assume?

In the first of our cases listed above the eye and nerves receive and focus the light, passing it to the cortex where it is viewed, but this process is complicated by our binocular vision, our roaming eye, and the fact of our brain's intelligent involvement with the interpretation of the light. In the second example we see a more pronounced case of the interpreter. As the artist sees, he/she is already filtering the visual input and it is again filtered in the translation onto the painting surface, this time perhaps more consciously. The rules for the mapping of 3D to 2D can become extremely complex.

One of the first fields to analyse the problem of projection was cartography. Travelling the seas and oceans, or travelling any great distance, requires accuracy in the plotting of distances and headings, so precise map making as been important to many peoples for a long time. As ship technologies advanced and knowledge of the entire globe became relevant, representing the surface of a sphere accurately on paper became a crucial subject.

"The transposistion of a sphere onto a plane is impossible without bending or tearing; [this] will lead to a deformation of the relative arrangement of the elements of this surface" (Bertin 1967 pg.288)

Despite the vital nature of the information on a map, for a long period the plotting of land masses was subject to perceptual influence and guess work, countries considered important could appear massive in comparison to their actual dimensions, and areas that the mapper knew little of shrank and lost detail, as show in Fig 3.1.1.

As the data available to the cartographer became more precise and the accuracy of the mapping more significant the techniques for presenting the information had to become more sophisticated. As stretching and tearing is unavoidable, the cartographers objective is to balance the shape accuracy against the scale accuracy. This is achieved by choosing bias and focus, making important areas precise and off loading distortions in less significant places, i.e. ice caps, canada.

Fig 3.1.1 left ancient-world global map right Goode's equivalent projection

This concept of bias and focus in the projected image is not some thing commonly found in work produced through 3D computer animation. Later we will see more examples of how projection can be altered to direct attention.

A related subject that will be utilised in section 4 is the texture mapping of 3D objects. Texturing a surface requires projection to move positional data from the three-dimensional object space to the two-dimensional parameter space so that 2D image files can be used to provide colour and other data for any point on the surface. In most systems polygonal models have parameter space coordinates U and V associated to each vertex. It is left to the user to apply projections and manually arrange the UV points on the 2D parametric plane, designing the mapping in the same way a map maker plans the layout of the globe. Here we see the same problems of cartographical projection, where best to lay the seams and how to distribute the distortion of the applied image.


          3.1.2 Mathematical projection

Commninos(2002 pg. 1) discusses the definition of viewing and projecting for computer graphics as a series of discreet transformations that concatenated allow the representation of an object-space point on a projection plane. The conceptual camera model used is shown in Fig 3.1.2.

Fig 3.1.2. virtual camera model as discussed by Commninos (2002)

As the projection transformation is separated from the other transformations the projection type is interchangeable. Commninos(2002 pg .6) makes the point that when representing a 3D object the user may wish to convey the general appearance or provide an image that enables the derivation of measurements. Those projections that convey the broad form of geometries are classified as perspective, and those that are utilised for measurement as parallel (due to the angle of the projection rays). This concept of differing projection types for differing needs supports the case for employing projection for emotive purposes. However it would be hard (even impossible) to classify and define more arbitrary projections mathematically as this would require the rules to be limited to those describable in the language employed by maths.

          3.1.3 Painters projection

Painters and other visual artists working in two dimensions have used projection as a device in their communication throughout history. It formed a fundamental concern in the painting and illustration of cultures world wide since they began recording life and thier world. Early examples may seem deficient in sophistication, but considering the skill in which artists and artisans employed the knowledge available to them with precise impact, we can perhaps see why naturalistic representation was not considered important. In European art sophistication in projection itself only appeared during the Renaissance, and in the Japanese and other eastern cultures restrained formal systems remained the principal style into modern times. Japanese and Chinese print works are particularly interesting for their use of axonometric projections, with parallel lines and unique view points. All artist are however free to subvert the rules the form they are using Piper (1984 pg.92) says that in practice it is unusual to find a painting deploying a single method systematically.
The Egyptian example below is from 1400 BC Thebes, and shows the mixture of projections exploited to present a garden scene. The vertical shapes (trees and ducks) are shown in side elevation and the horizontal pond is in plan. The contradictory orientations are unusual, but the essence of the scene is clear.

Fig 3.1.3 painting of garden from 1400 BC Thebes

The notion of representing multi-sensory knowledge of a subject, rather than the optical perception of it, lost importance in Western art as modernism arising from the Renaissance placed emphasis on scientific understanding. However as the world changed radically on entering the twentieth century and painting reformed (spurred by the arrival of photography) artists began to re-examine the notions of perspective, and looked back to the freedom of earlier models. The work by Chagall right shows a passage between projections across the picture plane that works in harmony with the floating figures.

   

Fig 3.1.4

   
   

Fig 3.1.5 Pitcher and violin

   

The movement principally responsible establishing the modern notion of freedom in viewing was cubism. The work of Picasso, Braque and Gris used multiple view points in factured single images to convey ideas of objects rather than simulate thier appearance. This could be seen as leading to the more ideas based art of the rest of the twentieth century. It is the subjective power of the cubist concept that could be beneficial to computer animation, a medium that being geometrically based may be well suited to its hard composistion, as opposed to the more fluid abstraction of painters such as Kandinsky.

Two examples of the use of multiple veiwpoints in commercial art are presented next. First is a sample page from the graphic novel 'Jimmy Corrigan, The Smartest Kid on earth'. Strip cartoons and comic books are intrinsically linked to the presentation of multiple view points and (in most narrative works) multiple time frames. By using formal conventions and unique devices the reader's focus is directed through the individual images, enabling the assembly of a complete space and event sequence in the mind. In the sample provided the point of view of the female character and her impression of the male based on his key features is eloquently shown in three vertical frames.
Below is a background panorama produced for the Disney Studios animation 'The Prince and the Pauper', a technique common in 2D cel animation.

   

Fig 3.1.6 Jimmy Corrigan The Smartest Kid on Earth. Ware, C.

   
These painted landscapes provide a convincing method for simulating the change in perspective associated with a moving camera that would not be possible through other drawn animation techniques, due to the prohibitive complexity of redrawing the scene. When filmed under a panning and rotating rostrum camera the illusion of the camera moving through the scene is achieved. Considered in their entirety, they provide an interesting expanded view.

Fig 3.1.7 panorama taken from 'The Prince and the Pauper'. The boxes indicate the framing when under a rostrum camera. If the panorama is moved so that the boxes are aligned under the rostrum in sequence with smooth interpolation between, a mobile camera is evoked. animated

       3.1.4 Perspective and its use

As stated previously the perspective projection was formalised in Europe during the Renaissance. It takes as its basis the visual fallacy that objects become smaller with distance from us, and the use of perspective in painting allows the artist to adopt a naturalistic style. The system of converging lines and vanishing points permits (with delicate skill) convincing illusions of three-dimensional space, and was well suited to the Renaissance artists admiration for the external world.

In computer graphics the emulation of perspective is widely studied and implemented. Having such a defined rule set, the technique of perspective projection was well suited to translation to computing machines. A definition of the perspective transformation is given by Commninos' notes (2002 pg.15). There are three distinct properties,

        the convergence of parallel lines,
        the diminution of size (i.e. foreshortening),
        non-uniform foreshortening.

Perspective projections are variable though, with vastly differing results obtain from changes in the number of vanishing points and changes in the viewing angle. Three point perspective is standard in computer animation implementations.
Although integrating computer generated elements into footage recorded with optical cameras is a widespread application of perspective rendering, there is a disparity between the processes of projection occurring in a camera lens and those of perspective projection (Kolb 1995). As the virtual camera is modeled as a pin hole it does not take into account the effects of the bending of light by the lens. These often have to be applied as post render processes.
So is perspective projection related more strongly to the human eye than the film camera? In many ways it is even less so. Human vision is binocular, perspective techniques are based on a single monocular viewing point. As discussed in the introduction to projection, the involvement of the human brain as an intelligent decipher agent complicates the assembly of the image, as does the constant motion of the eye. A vast body of visual knowledge is built from the rapid scanning of an environment. A single still perspective image is quite removed from this process.
Strict perspective is an idealised case of three dimensional perception and can be restrictive and weak as a subjective interpreter of a scene. Piper (1984 pg.95) states his objections:

"At a philosophical level it could be argued that optical perspective implies a narrowing of outlook, an over simplification of the complex and partly mysterious process by which we see the world, positioning as it does optical experience as the sole source of visual art and implicitly downgrading other kinds."

Fig 3.1.8

The power of compositions derived from perspective by painters such Raphael and Van Eyck is undeniable however, and as a rule to be broken it can be just as useful. In the painting above de Chirico emotes a dream space by presenting a rendering of perspective that does not congeal correctly when viewed. His converging lines are inconsistent, breaking the ease with which such scenes are normally read, and generating tension in the viewer.


3.2 Ideas of seeing and representing

We have seen how projection may affect the subjective properties of a representational image. What other factors related to the mapping of three dimensional forms onto a two dimensional medium are relevant to the objectives of this project? We shall briefly consider some ideas about representation that influenced the animated experiments that formed the second half of this project.

       3.2.1 Children’s symbolic sense of space

Young children have a distinct figurative style, a key feature of which is the symbolic but arbitrary scaling of picture elements. Size is strongly related to importance in their pictorial reasoning, and serves to communicate the child's ideas very efficiently.

"Around age four or five, children begin to use drawings to tell stories and to work out problems, using small or gross adjustments of the basic form to express their intended meaning." (Edwards 1993 pg 66)

Fig 3.2.1 The young artist has made the arm that holds the umbrella huge in relation to the other arm, because the arm that holds the umbrella is the important point of the drawing.

In computer animation the proportions of an object's appearance in screen space are commonly determined by its proportions in world space and the function of the perspective transformation (if we disregard any post render modification). Some research work has been undertaken into view dependant deformations of objects, and an animator/artist could develop a set of deformations to mimic symbolic scale relationships, but both concepts work in the geometric three dimensional world.

As this project was concerned with using world space to screen space relationships for subjective communication, it seemed well suited to explore the consequences of treating 2D scale as unlinked from 3D scale and as a signifier of importance. The ability to vary scale across an object would be interesting, and hopefully would help produce more engaging images.

       3.2.2 Incompleteness of image

When sketching real world objects and scenes, areas are often skipped over by the eye as it considers the overall structure, and the artist is free to skip over areas as he/she makes their marks. If a section of a shape is not vital to its general form, is it necessary to draw it? When viewing a drawing or painting gaps are read as filled as the brain has a strong capacity (and need) for closing lines and completing shapes. This incompleteness of image engages the imagination involving the viewer.

In some new form of cg projection, would it be possible to selectively portray a complete scene as incomplete at the stage of its rendering and translation to two dimensions? Again this seemed like a suitable question to explore in my experiments in this project.

Fig 3.2.2 Left two sketches showing varibable focus. Right J.M.W. Turner Château en France



       3.2.3 Continuity editing

Looking at film and other time based media, we can see that an elaborate code has developed to communicate spatial relationships that cannot easily or pleasingly depicted in one image. Continuity editing provides a structure for this as part of the common formal system.

"Continuity editing concentrates on the structuring of on and off screen space and on establishing and maintaining the viewers mental map" (Zettl 1999 pg 265)

In the development and investigation into a formal element concerned with spatial depiction, the integration with such techniques would be an important consideration.


       3.2.4 Hockney’s joiner works

In the early 1980s David Hockney became enthused by the possibilities of photography, a medium he had previously dismissed (see quote in introduction). He had begun to assemble large montages of multiple photographs to convey his impression of his surroundings. He wished to show more than he could through one solo photograph and wanted to overcome some of the problems discussed in section 3.1 that are he felt were inherent in photography. Greater depiction of time was another concern in Hockney's collages, stretching the stills camera beyond the frozen moment. In appedix 8.1 of this report there is a selection of his joiner works, showing their thematic development and the incorporation of time and narrative.

The use of multiple perspective based images to form one new subverted image in Hockney's joiners was a strong inspiration for my approach to this project. The concept of moving and expanding that technique to animation, particulary to a flexible and controllable form like cg, seemed to provide a strong basis for experimental work.


3.3 Issues relating to computer animation
       3.3.2 Work previously done in field of expanded perspective

As I mentioned in the introduction, there is an active research interest in more subjective styles of cg rendering, and in section 3.2.1 we raised the subject of view dependant deformations. There is however a much more closely related field of research. To key papers are presented in appendix 8.4 that look at the implementation and exploration of ray tracing based multipoint perspective rendering. Although biased towards the technical execution of such techniques they also asses the consequences of communicating information in this way.
Wood et al (1997) are concernd with the production of multipersective panoramas for 2D animation (such as the example seen in section 3.1.3). By utilizing the geometric capabilities of three dimensional computer animation vastly more complex panoramas than could be plotted than by hand are produced, although the best results are achieved by adding detail through a human drawing over the projection of simple shapes. As with the traditional panoramas, viewed in their entirety these images provide an interesting and engaging representation of a world.

Fig 3.3.1 an example of the system proposed in Wood et al (1997) and discussed in Advanced Renderman (pg.447)

Recently there as been a proposal for these kinds of raytraced multiview images as a narrative device. Andrew Glassner for Microsoft Research discusses the dramatic possiblities on his website and suggests techniques for the implementation using parametric curved surfaces as the base for his 'cubist cameras'.
Perhaps this will be a field that will see development as the more widespread adoption of NPR prompts greater analysis of the computer animation form.

___________________________________________________


contents

4 Tool Development

Having defined my desires in terms of finding a new kind of mapping from 3D to 2D, I was then required to build a technique and system that allowed the experimentation in and proving of my concept.
The chosen base for my technique was strongly related to the joiner photography works, in that I planned to generate multiple single point perspective image and combine them in larger assemblages. Hopefully, when taken as a whole these assemblages would present properties differing to those of single perspective renderings. I also wished to examine how certain elements of the form of computer animation (specifically its basis in time and ability to isolate and layer up elements) would interact with the technique of simultaneous multiple viewpoints.

It was not an aim of this project to produce a production ready distributable tool. A simple and functioning system was the only requirement to permit the testing of ideas, so even though some effort was placed in easing the generation of the assemblages for the user, the reader should be conscious of the experimental nature of the tool set.


4.1 Objectives for the tool

The base requirement for the implementation design was that it should allow the rendering and assembly of multiple individual viewpoints of a three dimensional computer scene.

Specific objectives were:

  • The number of viewpoints should be not limited and should be user definable.
  • The placement and orientation of the viewpoints should be user definable, and the user should be able to vary these over time.
  • The production of the rendered views from the viewpoint data base should not be highly inefficient.
  • The above tasks should be integrated with a general purpose 3D computer animation system.
  • The 2D assembly of the multiple images into a single image should be integrated with the viewpoint configuration stage, to enable the sharing of some of the viewpoint data base (e.g. no. of viewpoints)
  • The assembly should allow for the composition of layers of scene elements.
  •        4.1.1 Automation requirements

    It was decided that the generation of multiple virtual cameras within a scene and the individual rendering of each of these cameras would be a suitable method for producing the multiple images ready for assembly .SAY WHY.
    With a small number of cameras this could easily be achieved by hand, but to reach the desired complexity of projection and an arbitrary number of viewpoints (as specified in 4.1), an automated approach would be required.
    Asking the user to place one hundred cameras in a scene individually, then call the renderer one hundred times to generate the images, and finally to composite the hundred layers together may be acceptable once, but if repeated and varying tests are to be undertaken automation is required at all three of these stages.

           4.1.2 Need for absolute control

    As shown in the Hockney joiner works the subtle placement within and in relation to the entire montage is vital. In our system the user must have the option of over-riding every automated component of the process, be it the orientation of a camera, the specific position or rotation of the corresponding image in 2D, or the inclusion or exclusion of that image in the final collage.


    4.2 Scripted approach

    As the software to be developed was intended to be both functional and experimental, a flexible and speedy scripted approach seemed most suitable. A full fledged application, either as a plugin or stand alone program, would have required a greater devotion of project time and would have also been slower and more complicated to adapt as new ideas were tested.
    By using scripting languages to drive tools currently available /concepts could be evaluated quicker/. The reliance on pre-existing tools can however place limits on ambitions, but in the scope of this project none were met. A scripted approach can prove to lack efficiency, as the individual tools or functions called by the script are unlikely to be optimized for the specific task /one/ is putting them to. If the technique discussed in the remainder of this section was to be used in a production environment, the development of a full application would be necessary.

           4.2.1 Separation of tasks

    In analyzing the aims for the tool it be apparent that there were two distinct contexts in which the system had to operate. It was decided that these two contexts would benefit from separation, but with the sharing of data preserved.

    The generation of the multiple cameras seemed an operation best suited to the 3D interactive environment of the animation system (in the case of this project Maya), as this would allow the user to quickly assess and modify the location and orientation of the cameras in relation to the 3D scene they planned to render.

    The high memory and processor demands of the rendering task mean that it is best tackled outside of a demanding 3D application. It would be desirable for the rendering to be controlled from a simplified command-line based tool. It was also decided that by combining the rendering and assembly tasks into a single program they could share functionality.
    As the task hierarchy below shows, the rendering of the multiple cameras and the assembling of the rendered outputs are still considered separate. It may be that we wish to render and assemble in one go, but consider the case where the user modifies his cameras and outputs new data from the camera generation script. This new data can be used to assemble previously rendered images (we shall see why and how this may be useful later). Here we wish only to run the assembly section of the script utilizing the newly output data, but if we were forced to re render at the same time the whole process would be extremely wasteful.

    Fig 4.2.1. Hierarchy showing division of tasks

    Maya's internal scripting language MEL was chosen to generate the multiple virtual cameras.

           In overview, the MEL section of the system:
  • takes a polygonal mesh as an input
  • analyses the UV layout in the parameter space of the surface
  • attaches a camera to each vertex of the surface
  • and outputs ordered data for each vertex/camera to a text file
  • The system scripting language Perl was chosen to manage the rendering of the multiple cameras and the generation of the assembled composite. Perl was selected for its command line basis, its strong text and file handling, and its simplicity of form.

           In overview, the Perl section of the system:
  • reads the file created by the camera generation script, retrieving the total number of cameras and the data about the vertices/cameras
  • using this data and user provided parameters
  • then either
  • calls the Maya renderer for each camera in the specified scene file
  • then generates a Shake composite script describing the arrangement and layering of all the rendered images
  • or
  • just generates the Shake composite script
  • In the next two sections we will examine briefly the implementation of the two key tasks, looking at what problems required addressing and how their solution was approached.

    4.3 Camera layout and control

    The majority of the design of the final assemblage render is undertaken in the 3D environment, with only minor tweaking performed in the Shake composite. The use of the 3D system has three benefits:
  • it provides a metaphor to aid the mental mapping of the three dimensional scene to the 2D arrangement.
  • it allows the artist to make use of familiar geometric and texture mapping tools.
  • it allows the viewpoints to be seen in relation to the objects in the scene.

  •        4.3.1 Making use of Maya’s tools to provide an environment for camera design

    We use a polygonal mesh to serve as a control template for the placement and aiming of the cameras. A polygon object had the properties needed for camera design in each of its vertices:
  • location is provided by the world space positional coordinates of a vertex
  • orientation is provided by the normal vector of each vertex
  • eventual placement in the 2D composite of the rendered image associated each vertex/camera is provided by the UV coordinates associated with each vertex
  • The user generates a polygon mesh and begins planning the placement of viewpoints in his 3D world. Vertices can be added and removed, and translated.
    The utilisation of the UV data of each vertex is key to providing control over the mapping from 3D to 2D. As discussed in section 3.1, UV mapping for texturing provides a projection from object space to 2D parameter space. Maya has many built in tools to construct UV projections and allows the individual adjustment of UV locations. This allows the user to design and visualize the layout of the rendered images inside the 3D environment.
    UV placement is quick and easy in Maya, where as adjusting multiple Pan nodes in shake is not. The camera generating script writes the U and V values for each vertex to a file, and these values are used by the Perl script to set the X and Y values of a Pan node for each tile (as we shall call the rendered images output from each of the multiple cameras from now on).
    As stated earlier the vertex normals provide orientation for the cameras. This is achieved by the setting in the script of a normal constraint for each camera. The automated vertex normals calculated by Maya through interpolation of the face normals were found to often clash with the orientation desired for the cameras. Fortunately Maya permits the user to set each vertex normal absolutely, over ruling the default ones. This lets the designer of the camera set to aim specific cameras at specific focal points.

    Positional data for each camera is set via expressions, which are generated by the MEL script. The expressions query the world space coordinates of the camera's corresponding vertex on the base geometry. These coordinates are then assigned to the transformation values for the indexed camera. Expressions are used so that the values are evaluated each frame, permitting animated and deforming base geometries.

       

    Fig 4.3.1. script GUI in Maya

       

    A later addition to the camera generation script was the consideration of point proximity. In the vertex loop of the script, each UV is considered against all the others in the object and the most proximal is found using pythagoras. The distance to the closest UV is assigned to the vertex as a colour value for storage. The result is a surface coloured white in the most concentrated areas and black in the more sparse areas. These values are then used as scaling factors on the cameras' field of view, having the effect of wider viewing cameras in sparser areas of the geometry. The scaling data is also written to the file that contains the U and V values, and is utilised in Shake as a 2D tile scaling factor (as will be seen later).

    A very simple GUI was developed to ease the execution of the script. It provided a mechanism for specifying the file for the vertex data to be written to, and another for setting the base level for the cameras' field of view. More parameters for the script could be set in this manner if the project was expanded in any way.

    The MEL code can be found in the appendices for those interested in studying the algorithms used to effect the camera placement


    4.4 Rendering and tile assembly

    The principal objectives for the Perl rendering and assembly script are:

  • to call the Maya renderer to render the specified scene file repeatedly until all the cameras it contains have been rendered through
  • and to then compose a shake compositing script to overlay the generated tiles in the correct arrangement.

    The number of cameras to be rendered is written to the data transfer text file by the MEL script at the camera generation stage. This number is read by the Perl script, and rendering is achieved through a for loop.

    in pseudo code
           for (i=0;i < total_number_of_cameras;i++)
           {
           render this_scene.ma with cameraNumber_i to output file tile.i.tif;
           }

    The render command accepts a frame range too, so animated tiles are accommodated.

    The assembly section of the script is more involved, but works in a related manner. To begin a blank .shk Shake script file is generated. Shake script files are ascii text descriptions of the nodes and connections required to perform a composite using the Shake engine. The Perl program then prints into this the nodes (and their parameters) that should appear in the final composite tree. Shake was chosen for this task because of its open nature, allowing us to perform complex compositing operations with out using the graphical UI.
    A global control node is entered into the description of the tile assembly, with variables attached that will be used in Shake expressions (also produced by the script) to vary parameters of all of the tile nodes. For example a UV scaling factor is provided that permits the scaling of the X and Y transformations applied to all tiles, through one slider. (see the basic control interface ini fig 4.3.2)

    A background and then all of the tiles are entered as Filein nodes. A for loop controls the repetition of the tile entries, and it is simple to append other nodes to the Filein during this control structure. A generic transformation is appended to each tile, with expressions to perform the translations in X and Y. The values for the X and Y displacements are read in during this loop, using the U and V data from the data transfer text file. The proximity value is also read in and is utilized in the scaling expression. The amount to which this proximity scaling is applied can be controlled from the global control node, to provide the option of tweaking its visual influence. Other nodes are appended for aesthetic adjustment, and to provide greater positional control. Finally in the loop an Over layering node is appended, ready to receive the next tile as its second input.

       

    Fig 4.3.2 tile arranging system as it appears in Apple's Shake compositor

       
    The result of the assembly script section is a completed Shake compositing script containing the complete set of rendered tiles in one assemblage, ready for viewing, adjustment, or rendering.

    The Perl script code can be found in the appendices.

    ___________________________________________________


    contents

    5 Exploration of Consequences


          5.1 Tests run parallel with tool development

    In order to assess the functionality of the system as it was developed visual test cases were necessary. Short animated sequences were produced as the system was progressively modified and its capabilities improved. As well as providing visual evidence of success or failure, these tests also gave the first indicators of the aesthetic consequence of the technique.

    Single frames from some of these tests are presented here and in a library of the animations in appendix 8.3

    Fig 5.1.1

    This image shows the first visual output from the system. A basic coloured and numbered grid room was employed to provide ease of assessment, not aesthetic pleasure, yet the potential of the technique is clearly evident.
    A large degree of hand arrangement was employed in this assemblage, which puts it in contrast with the later tests.

    One discovery made at this stage was that when viewing the assembled tiles, the eye sought out the disparities between and the edges of the individual images. This could act against the perception of the projection as a whole.

    Fig 5.1.2

    In this next sequence the differences between the tiles is exaggerated to make evident the fractured nature of the picture. Colour correction varies between tile, they are randomly scaled, and subtle drop shadowing has been added to emphasize the edges. These adjustments made it easier for the viewer to accept the composite basis of the animation and added to the visual sumptuousness. This test also presents the first rendering of an actual scene (an early example of the bathroom employed in the final product). We can begin to appreciate the unique view of an ordinary subject that can be achieved with the technique.

    Fig 5.1.3

    The use of a regular grid type arrangement was studied next. These tests considered and explored the affect of decreasing the connection between the 3D position of the cameras and their 2D placement. The results are strongly abstracted impressions of the room. This was felt to be moving close to the limits of this projects' scope before the first principals had been fully understood.

    The next two examples were generated in the stages leading to the production of the final animation sequences. They were used to examining two possibilities for inclusion in those final animation pieces.

    Fig 5.1.4

    First is a study of a deforming camera base. Previous to this the cameras had been moved as a group in 3D and this as a uniform motion in the 2D tiles. By animating a change in the base shape, the relative motions are non uniform across the image. The emotive consequences are not clear in this sequence, other than the sense of intrigue such a special projection produces.

    Fig 5.1.5

    Last is a consideration of element layering as was suggested as an objective in section 4.1. It was hoped that by treating individual elements as separate assemblages and then layering these assemblages together a distinct new visual could be produced. Hockney limited his joiners to whole photographs, wary of breaking the boundary edges, but if we have the ability to treat objects individually we may be able to help unify them in the final image. Their fractured nature would be retained but the relationships between the objects of the scene could be altered/emphasized by the various combinations.
    In this example the figure and the shower curtain are assembled with themselves on their own layer, and the room on its. The two assemblages are then layered together, but as the figure is on his own layer he remain distinct.

    To the library of animations.


    5.2 Production of animated sequence

    It was decided to produce a short animated sequence to act as a proof of concept, a test pad for further exploration, and as an output to be evaluated.

          5.2.1 Theme of animation in relation to theme of project

    A study of a human figure in relation to their environment was chosen as the theme for the animated work. The strong connection and understanding people posses for other people would help engage the viewer, and humans and their behavior are visually interesting in terms of colours, shapes and motions. As this project was concerned with the investigation of multipoint perspective in a time based medium, motion would be important. It was hoped that exploration of the temporal and causal consequences of the technique would be possible in the animation, possibly by the shifting of individual tiles in time.

    The environment of the bathroom was chosen for its variety of textures and colours, and the interesting but simple shapes that it was constructed from. It was modeled and textured in a limited manner, which considering the complex way in which it was to be rendered, seemed suitable.

    Fig 5.2.1 bathroom environment

    A figure from a previous project was used to act in the scene. His polygonal quality matched well with the obvious construction of the assemblage images.

          5.2.2 objectives for animation

    As stated above, it was intended for the animation to experiment with time shifting of tiles. Other specific objectives for the use of the multi-viewpoint technique in the final animation were:
          enhancing the portrayal of mood
          depicting the relation between the character and the environment
          presenting an expanded image of the environment
          employing selective inclusion and scaling of tiles to draw focus around the image
          examining the general effects of viewing a world through this device

          5.2.3 Production limitations

    Unfortunately the time available for the final animation test was restricted. It was decided to limit the piece to the last five of the objectives, as the exploration of the temporal issues appeared to be a large and fundamental issue requiring sizable attention in isolation before integrating it with the other subversion techniques.

    ___________________________________________________


    contents

    6 Evaluation


    In watching the final animation sequence we can view and assess many of the unique visual properties resulting from the new multiview technique. This visual exploration of the base principles meets the aims set out at the beginning of this report. Does this mean that the project has been a complete success?

    6.1 Suitability for narrative, expression, or abstraction?

    As the animation eventually produced was limited in duration and in subject matter, the assessment of the new projection's suitability to these three domains is difficult to conclude.
    The system tests were restricted to examining the interiors of cubic rooms. Even more exciting results may have been achievable from considering arrangements of positive forms in an empty space, or by looking at landscaped.
    The narrative content of the piece is constrained by its short duration. In a longer work I would have wished to consider more fully the interaction between the simultaneous presentation of multiple events and viewpoints, and the system of continuity editing.

    Abstraction was approached in the tests that examined the disconnection of the 3D camera position and the corresponding 2D tile placement. Similarly, two of the other test pieces were accidentally laid out in reverse U order, yet they maintained a notion of the location they were studying. Small glimmers of the bathroom were visible within the more general abstract pattern. This field would perhaps be an excellent use for the projective technique, but as said previously, I felt it to be beyond the concerns set for this project.

    Expressive use was the most explored of the contexts, and i feel that it has been the area that has shown the most success. By utilizing the devices of focus arising from the multiview techniques I have been able to generate a compelling atmosphere in the animation produced. Again however the range explored (in this case, of emotions) has been limited. It may be that the technique has an intrinsic emotion that it conveys, rather than being adaptable. This would be an interesting preposition to explore in a future project.

    6.2 Are the images compelling or confusing?

    This question had been a concern present since this project was undertaken. I was anxious that too radical a change in the projection mechanism would result in images that were off putting in their complexity, and removed from the viewers concepts of perception.
    The final animation pieces appear to reject these worries, as I believe they communicate a strong notion of their environment any sense of confusion in the viewer is converted to intrigue and engagement.
    It would have been extremely useful to have more primary evidence to support the above statement. Presenting to a test audience a direct comparison between a conventional rendering of a scene and a rendering using the assemblage technique could have given useful and interesting feedback. This is something that will be taken into consideration in future experimental work to aid a more objective evaluation.

    6.3 Project assessment

    I believe that the aims and objectives defined for this project were met in its execution, but not to the degree that I had hoped for. This was perhaps due to an over estimation of the amount achievable in the time available, and the consequent ambitious scope set for the work. The high ambition is reflected in the number of issues proposed for further consideration.

          Assessment of objectives

    Develop an understanding of projection and perspective representation in relation to painting and animation.
          This portion of the project was thoroughly considered and I believe the case considered for my new technique is strongly supported by my research. However, it is also evident that there was a bias in the research, and that this affected the range possessed by the final animation. In the contextual research, issues in still images and media were the primary focus. I would have liked to have addressed more completely the consequences of a time based medium such as animation.

    Produce a tool set integrated with the systems 'Maya' and 'Shake' to allow the rendering and assembly of multiple viewpoint image sequences.
          The system developed met the tasks given to it and enabled the completion of my animation product, The technique employed did however prove to lack speed and efficiency at the rendering stage, as it required the entire scene to be reloaded into memory from disk many times. This lack of speed hampered the production of the animation, but this is true of most rendering.

    Execute a short animation exploring any unique aesthetic and temporal properties possible using this device.
          The limits and successes of the animation have already been discussed, but to conclude I can state that I was able to generate a sequence that demonstrates the potential for the technique, if not all the possible consequences.

    6.4 Specific futher developments

          Screen size
    In the finished animation it was decided to view the assemblages by panning across their surface, partially to explore this extra level of directorial control, and partially to allow the viewer to see enough of the contents of the tiles. Perhaps the affect of the assemblages as a whole would have been more interesting. Hockney's joiner works were large items, and the viewer was allowed to consider its portions at will, rather than having an order forced upon them as the panning does.
    My assemblages may be better suited to a larger format such as the cinema screen or projection in a gallery.

          2D orientation of the tiles.
    This was not controlled in any way by the scripted system, and was left to be hand set by the artist. This allowed them to place more of their mark on the work, but perhaps this could be better controlled.
    Finding a solution to the 2D orientation would provide and interesting technical extension to the project.

    ___________________________________________________


    contents

    8 Bibliography and Acknowledgements

    'Drawing on the Right Side of the Brain'. 1993. Edwards,B. Harper Collins.
    'Sight Sound Motion'. 1999. Zettl,H. Wadsworth.
    'The Joy of Art'. 1984. Piper, D. Mitchell Beazley Publishers.
    'Multi-Perspective Images for Visualisation'. 2002. Vallance,S., Calder.P. Proceedings of VIP 2001.
    'Multiperspective Panoramas for Cel Animation' 1997. Wood,N et al. Proceedings of AMC SIGGRAPH 1997.
    'A Realistic Camera Model for Computer Graphics' 1995. Kolb,C. et al. AMC SIGGRAPH 1995. 'Postmodernism at NoPatAnswers.com' http://216.157.49.208/ray/postmodernism.htm
    'Visual semiotics - perspective workshop' http://courses.washington.edu/hypertxt/cgi-bin/12.228.185.206/html/pmontage/perspecdemo.html
    'David Hockney' 1987. Livingstone,M. Thames and Hudson Ltd.
    'Advanced Renderman'. 2000. Apodaca, A.A., Gritz, L. Academic Press.
    'The Renderman Companion'. 1989. Upstill, S. Addison Wesley.
    'Maths for Computer Graphics 2 course notes, section 7 Viewing and Projection Transformations'. 2002. Commninos, P. Bournemouth University.
    'PERSPECTIVE International Media Art and Media History Exhibition' http://www.c3.hu/perspektiva/
    'Semiology of Graphics'. 1967. Bertin, J. University of Wisconsin Press.
    'Jimmy Corrigan, or The Smartest Kid on Earth'. 2001. Ware, C. Random House.

    Thanks to - ian mackinnon, phill allen, steve bell

    ___________________________________________________

    8 Appendices         < to the library of images, source code, production examples, and reference papers

    ___________________________________________________

    joiner footer