SUMMER 2021
APP-LESS XR
I had the pleasure to direct this project with ustwo Studios in New York City. We explored an app-less system UI that broke apart apps and surfaced their functions and data on a system’s top level, primarily for an XR headset device.
Reframing data
We used the metaphor of malleability within XR to describe a mental model of how data could be physically manipulated. Any piece of data within the system UI could become an object, malleable, editable, and placed into larger functions or structures.
CORE FUNCTION
We defined two key organizational objects: structures which hold and help reorganize data based on user behavior, and tools, which are physicalized extractions of popular functions within apps.
Structures
3 primary structures focused on productivity, communication, and life organization.
Pegboards
Structures to help organize widgets and data based on a specific task or workflow
Amoebas
Structures to share content and applications among multiple users
Calendar
Structures to organize data and activities based on time
These Hololens demos showed how a user could decorate a space with digital objects, as well as create their own custom widget boards connected to their IoT devices
These demos and illustrations show how sharing music, messages, media, and personal screens could work and feel within an Amoeba
Each date on the calendar serves as a removable shelf that contained all the content needed for the day
Tools
Alongside structures, I did a personal investigation on digital tools which are extracted functions from system UI that are surfaced to the user as AR objects.
Stamp Tool
functions as an extension of the clip board, allowing people to copy and paste data effortlessly, or in more creative applications, duplicate 3D objects or collect 3D assets from other metaverse spaces.
Knife Tool
functions as a tool to splice, cut, or edit and changes its function based on the file type. In the example above, when the knife tool is used to make an incision on a video timeline, the tool automatically exports a screen capture or gif based on the number of cuts.
Pin Tool
could aggregate functions across various apps that centered around location data. Dropping the pin tool on a busy corner in Chelsea would provide information to rideshare apps, restaurant reviews, as well as spatially leaving comments for friends
These illustrations investigated how tools could be managed and stored. The image above shows a digital shelf that could be invoked with a special gesture or button
Above shows a pop-up menu that could be accessed with a special gesture.
This last image illustrates a power-glove like ergonomic design where the tools are tethered to the user’s hand for easier access.
Microsoft Hololens Demo
A fellow CT and I created a demo to show how structures and tools could work. The demo is built in Unity Engine and I helped create the 3D assets, materials, and visual effects.
RECAP
PROBLEM
Tasked with working with a design agency to investigate new interaction paradigms in the world of XR.
Research
Conducted research with content creators to better understand their needs which revealed that editing content was the most painful part of the process.
Impact
Dramatic shift of organizational philosophy about how interface designs for new form factors shouldn’t be direct ports of existing paradigms.
AI HOME
Home
Patents and Awards
About
SUMMER 2021
APP-LESS XR
I had the pleasure to direct this project with ustwo Studios in New York City. We explored an app-less system UI that broke apart apps and surfaced their functions and data on a system’s top level, primarily for an XR headset device.
Reframing data
We used the metaphor of malleability within XR to describe a mental model of how data could be physically manipulated. Any piece of data within the system UI could become an object, malleable, editable, and placed into larger functions or structures.
CORE FUNCTION
We defined two key organizational objects: structures which hold and help reorganize data based on user behavior, and tools, which are physicalized extractions of popular functions within apps.
Structures
3 primary structures focused on productivity, communication, and life organization.
Pegboards
Structures to help organize widgets and data based on a specific task or workflow
Amoebas
Structures to share content and applications among multiple users
Calendar
Structures to organize data and activities based on time
These Hololens demos showed how a user could decorate a space with digital objects, as well as create their own custom widget boards connected to their IoT devices
These demos and illustrations show how sharing music, messages, media, and personal screens could work and feel within an Amoeba
Each date on the calendar serves as a removable shelf that contained all the content needed for the day
Tools
Alongside structures, I did a personal investigation on digital tools which are extracted functions from system UI that are surfaced to the user as AR objects.
Stamp Tool
functions as an extension of the clip board, allowing people to copy and paste data effortlessly, or in more creative applications, duplicate 3D objects or collect 3D assets from other metaverse spaces.
Knife Tool
functions as a tool to splice, cut, or edit and changes its function based on the file type. In the example above, when the knife tool is used to make an incision on a video timeline, the tool automatically exports a screen capture or gif based on the number of cuts.
Pin Tool
could aggregate functions across various apps that centered around location data. Dropping the pin tool on a busy corner in Chelsea would provide information to rideshare apps, restaurant reviews, as well as spatially leaving comments for friends
These illustrations investigated how tools could be managed and stored. The image above shows a digital shelf that could be invoked with a special gesture or button
Above shows a pop-up menu that could be accessed with a special gesture.
This last image illustrates a power-glove like ergonomic design where the tools are tethered to the user’s hand for easier access.
Microsoft Hololens Demo
A fellow CT and I created a demo to show how structures and tools could work. The demo is built in Unity Engine and I helped create the 3D assets, materials, and visual effects.
RECAP
PROBLEM
Tasked with working with a design agency to investigate new interaction paradigms in the world of XR.
Research
Conducted research with content creators to better understand their needs which revealed that editing content was the most painful part of the process.
Impact
Dramatic shift of organizational philosophy about how interface designs for new form factors shouldn’t be direct ports of existing paradigms.
AI HOME
Home
Patents and Awards
About
SUMMER 2021
APP-LESS XR
I had the pleasure to direct this project with ustwo Studios in New York City. We explored an app-less system UI that broke apart apps and surfaced their functions and data on a system’s top level, primarily for an XR headset device.
Reframing data
We used the metaphor of malleability within XR to describe a mental model of how data could be physically manipulated. Any piece of data within the system UI could become an object, malleable, editable, and placed into larger functions or structures.
CORE FUNCTION
We defined two key organizational objects: structures which hold and help reorganize data based on user behavior, and tools, which are physicalized extractions of popular functions within apps.
Structures
3 primary structures focused on productivity, communication, and life organization.
Pegboards
Structures to help organize widgets and data based on a specific task or workflow
Amoebas
Structures to share content and applications among multiple users
Calendar
Structures to organize data and activities based on time
These Hololens demos showed how a user could decorate a space with digital objects, as well as create their own custom widget boards connected to their IoT devices
These demos and illustrations show how sharing music, messages, media, and personal screens could work and feel within an Amoeba
Each date on the calendar serves as a removable shelf that contained all the content needed for the day
Tools
Alongside structures, I did a personal investigation on digital tools which are extracted functions from system UI that are surfaced to the user as AR objects.
Stamp Tool
functions as an extension of the clip board, allowing people to copy and paste data effortlessly, or in more creative applications, duplicate 3D objects or collect 3D assets from other metaverse spaces.
Knife Tool
functions as a tool to splice, cut, or edit and changes its function based on the file type. In the example above, when the knife tool is used to make an incision on a video timeline, the tool automatically exports a screen capture or gif based on the number of cuts.
Pin Tool
could aggregate functions across various apps that centered around location data. Dropping the pin tool on a busy corner in Chelsea would provide information to rideshare apps, restaurant reviews, as well as spatially leaving comments for friends
These illustrations investigated how tools could be managed and stored. The image above shows a digital shelf that could be invoked with a special gesture or button.
Above shows a pop-up menu that could be accessed with a special gesture.
This last image illustrates a power-glove like ergonomic design where the tools are tethered to the user’s hand for easier access.
Microsoft Hololens Demo
A fellow CT and I created a demo to show how structures and tools could work. The demo is built in Unity Engine and I helped create the 3D assets, materials, and visual effects.
RECAP
PROBLEM
Tasked with working with a design agency to investigate new interaction paradigms in the world of XR.
Research
Conducted research with content creators to better understand their needs which revealed that editing content was the most painful part of the process.
Impact
Dramatic shift of organizational philosophy about how interface designs for new form factors shouldn’t be direct ports of existing paradigms.
AI HOME
Home
Patents and Awards
About