WebGL Surfing through the third dimension - MOVES Institute

1 . SpiderGL A JavaScript 3D Graphics Library for Next-Generation WWW Marco Di Benedetto, Federico Ponchio, Fabio Ganovelli, Roberto Scopigno Visual Computing Lab – ISTI – CNR 

2 .3D Content and the WWW  Many attempts have been done so far  ActiveX objects  Proprietary browser extensions or plug-ins  Most libraries/plugins are too high level to directly access underlying graphics system  Developers forced to comply to some paradigm (scene graphs)  CG experts often find limitations due to high-level abstraction  Standardization needed  WebGL : JavaScript bindings to OpenGL|ES 2.0 2 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

3 .Outline  Introduction to the WebGL Rendering Pipeline  The SpiderGL Library  Case Study: Mesh Rendering  Library Features & Demos  Conclusions & Future Work 3 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

4 .What is WebGL  Actually what will be (standardization in progress)  Specification owned by the Khronos Group  Supported (and defined) by all major web browsers devs  JavaScript bindings to OpenGL|ES 2.0  Almost 1-to-1 mapping, some modifications to meet JS developers’ habits and security issues  Enables home computers and mobile devices to natively access 3D content directly from web pages (no external plug-ins) 4 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

5 .OpenGL|ES 2.0  Stripped version of OpenGL, focused on Embedded Systems  Programmable : NO fixed function pipeline  Immediate mode (glBegin / End)  Use vertex / index buffers  Transform Stage (matrix stacks)  Explicit shaders uniforms  Lighting (lighting model, light sources, materials)  Lighting computation through shaders code  Named vertex attributes (glVertexPointer(…), …)  Generic attributes through glVertexAttribPointer(index, …)  Data entirely resides on GL resources (buffers, textures)  Buffer Centric API  Restrictions  Texture formats, data types, shading language limitations, ... 5 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

6 .OpenGL|ES 2.0: Architecture (simplified) Vertex Attributes Stream Index Stream Data VB IB TEX RB FB 0 … N-1 Memory Code Vertex VS FS UNI Vertex Puller Puller Memory Vertex Processor Primitive Primitive Assembler Assembler & & Fragment Processor Rasterizer Rasterizer Texture Units Uniforms 0 … M-1 Framebuffer Output Output Merger Merger Pixel Pixel Operations Operations Screen 6 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 Screen 

7 . OpenGL|ES 2.0: API Structure  Context  Capabilities Query  Error Query  Creation / Activation / Destruction not part of specifications (EGL for this) Pipeline  Objects Configuration  Resources creation / edit / bind / destruction &  Data Set & Get Data  Configurable Stages  Enable / Disable  Constants / Parameters Set & Get  Programmable Stages  Accept user-defined programs (shaders)  Vertex Pipeline  Geometric primitives draw (activate Vertex Puller) Execution  Pixel Pipeline  Framebuffer clear & readback (activate Pixel Operations)  Synchronization  Command Buffer flush & wait 7 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

8 .Motivations: Bring CG to the WWW  WebGL as a link between Web and CG developers  But... it’s very low-level!  Gaps to be filled:  Speed, of course  Algebraic and Geometric structures & algorithms  Asynchronous data fetch facilities  Assets loading  Ease the use of WebGL  SpiderGL aims at filling these gaps 8 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

9 .SpiderGL: 3D Graphics for WWW  Lightweight LGPL library ( http://spidergl.org )  provides typical CG data structures & algorithms  Sits on top of WebGL for realtime rendering  Goals:  Efficiency  asymptotic bounds are not the only concern  Simplicity & Short Learning Time  Key for fast application coding  Flexibility  Help with common tasks and create a robust middle layer for more complicated designs 9 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

10 .Leading desing considerations  Using a third-party library should be easy  Do not impose any design choice a priori  Avoid over-abstraction  In CG simple and self-contained types works very well  Users must be able to reuse as much as possible of their former knowledge on the subject  All the entities have de-facto standardized names and behavior  Experienced users often want fine control  low-level access 10 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

11 . SpiderGL Architecture  GL: rendering  Mesh: editable or renderable 3D model  Space: geometric s. & a.  Async: asinchronous data requests managing  UI: user and html interface 11 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

12 .Case Study: Mesh Rendering  How a typical 3D model is represented:  A list of vertices (each vertex is a bundle of data)  A list of vertex indices, specifying how they are connected to form basic geometric primitives (points, segments, triangles) Vertex & Index Buffers  What we need to draw it  POV intrinsic and extrinsic parameters  Some procedure to make light sources and model material interact to form colors on the final image Shaders & Uniforms 12 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

13 .Meshes in SpiderGL  Mesh Structure  Set of named vertex attribute streams (position, normal, color, custom, ...) with NO predefined semantic  Set of named array or indexed primitive streams (points list, triangulated surface indices, wireframe segments, ...)  Mesh twins:  SglMeshJS: an editable data structure  SglMeshGL: a renderable graphics resource  Unified interface, conversion functions 13 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

14 .Create a Renderable Mesh /*******************************************************************/ var mesh = new SglMesh(gl); // add a vertex attribute named “position” (type is inferred) mesh.addVertexAttribute("position", 3, nativePositions); // add a NORMALIZED vertex attribute named “color” mesh.addVertexAttribute("color", 3, nativeColors, true); // add connectivity information named “triangles” (type is inferred) mesh.addIndexedPrimitives("triangles", gl.TRIANGLES, nativeIndices); // add “array” primitives named “vertices” mesh.addArrayPrimitives("vertices", gl.POINTS, 0, 3); /*******************************************************************/ 14 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

15 .Create a Shader Program /*******************************************************************/ // vertex shader source code string var vertSource = ""; vertSource += “uniform mat4 u_mvp; \n"; vertSource += "..."; vertSource += "attribute vec3 a_position; \n"; vertSource += "attribute vec3 a_color; \n"; vertSource += "..."; // fragment shader source code string var fragSource = "..."; // program setup var prog = new SglProgram(gl, [vertSource], [fragSource]); /*******************************************************************/ 15 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

16 .Stream Mapping  The mesh has 2 vertex attribute streams (position & color)  The vertex shader has 2 input vertex attributes   A correspondence between the two sets must be established /**********************************************************/ ……… mesh.addVertexAttribute("position", 3, nativePositions); mesh.addVertexAttribute("color", 3, nativeColors, true); ……… /**********************************************************/ /**********************************************/ ……… vertSource += "attribute vec3 a_position; \n"; vertSource += "attribute vec3 a_color; \n"; ……… /**********************************************/ 16 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

17 .Rendering /*******************************************************************/ // <name : string>  connect shader attr “name” to mesh attr “string” var streamMapping = { a_position : "position", a_color : "color" }; // <name : value>  set program uniform “name” to value “value” var uniforms = { u_mvp : getModelViewProjectionMatrix() }; // utility function (full control available through SglMeshRenderer) sglRenderMeshPrimitives(mesh, "triangles", prog, streamMapping, uniforms); /*******************************************************************/ 17 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

18 .Option Parameters  The GL module aims at simplifying common WebGL tasks  Creation of WebGL objects  Textures, programs, framebuffers, ...  Common sequence of several commands  SpiderGL offers reasonable but overridable defaults /*********************************************************/ var textureOpts = { minFilter : gl.LINEAR_MIPMAP_LINEAR, generateMipmap : true, onload : function () { ... } }; // create texture from remote image var tex = new SglTexture2D(gl, "image_url", textureOpts); /*********************************************************/ 18 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

19 .Dealing with WebGL Objects  Contstructors: SpiderGL provides easy-to-use configurable functions to create WebGL Objects  Wrappers: each native handle can be wrapped by optimized high-level objects  Flexibility: experienced users may want direct low-level control:  obj.handle  native WebGL object  obj.synchronize()  update state after low-level usage 19 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

20 .Overcoming Mesh Limitations  Ex.: OpenGL|ES 2.0 (and so WebGL) does not allow 32 bit vertex indices   Limited to 64K vertices per chunk  SpiderGL uses packed-indexed primitive stream to seamlessly allow very large meshes  Mesh is automatically subdivided into sub-meshes 20 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

21 .On the Math / Geometry Side  Support CG-related linear algebra entities  2, 3, 4-dimensional vectors, 2x2, 3x3, 4x4 matrices, quaternions  Two choices: operate on native JS arrays or boxing objects  Standard geometric entities  Planes, spheres, boxes, ...  Intersection queries  Transformation Stack  Editable Mesh (SglMeshJS)  Several (and more coming) algorithms on trimeshes 21 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

22 .Assets, Asynchronous Fetch and UI  COLLADA, OBJ, JSON (and more coming) importers  Every kind of handled document can be loaded with a uniform asynchronous request interface  Images, 3D Models, shaders code, ...  Dynamic priority queues for multiresolution  Requests can be interrupted, priority can change, transfer callbacks can be installed  Special watcher object for completion of batched transfers  GLUT-like interface and Interactors  first-person camera, trackball, ... 22 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

23 .What can be done with SpiderGL I’m tired up here... Let’s see some demos! http://spidergl.org/code.php 23 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

24 .SpiderGL to Assist Content Creation  Multimedia Web repositories are widespread  Images, Video, Audio, Text, ...  What about 3D Repositories?  WebGL will bring HW Accelerated 3D Graphics to WWW  Lots of 3D databases  We need an effective way to deploy content to users  MeShade: a content authoring tool  Let’s have a look!  http://spidergl.org/meshade/index.html 24 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

25 .Conclusions  SpiderGL as a Geometry processing and Visualization library  Do not over-abstract graphics objects, ease the completion of common task, allow low-level access  Real-time performances with WebGL with several case studies  Future Work:  Continuous improvements, growing set of algorithms  Toward an automated process to make high-end 3D content to be available to handheld devices 25 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010 

26 . Thank You! http://spidergl.org Questions? marco.dibenedetto@isti.cnr.it 26 Los Angeles Di Benedetto et al. - SpiderGL: A JavaScript 3D Graphics Library for Next-Generation WWW - Web3D 2010