Nansense

This page contains information about how to get started using the Nansense suits, and how to record and access recorded data. We also provide tips for troubleshooting.?Documentation from Nansense can be found here (ask Kayla for login info): https://www.nansense.com/docs/

Assembling the Nansense suit

The suit contains five strings of sensors that must be placed correctly in order for the suit to work.?Please refer to the diagram for the placement of sensor strings (click to see larger version in another window). The diagram shows the suit from the back without the battery or hub. To reach some of the connection points (see below), it is helpful to flip the suit over so it lies on its back and the inside (unzipped) is accessible. We recommend placing the strings in the following order: left arm, right arm, thoracic, left leg, right leg. Sensors should be oriented with the screws facing away from the body.

A few pointers:

• The sensors sit on the foam platforms that are sewn into the suit. Make sure the sensors are relatively centred on their platforms, since their position on the body needs to be quite precise.
• There are small holes (connection points) marked as purple spots on the diagram.?Getting the sensors through these holes can require persistence and patience. Don¡¯t tug on the cables, since they are fragile.
• The suit should be assembled properly with all of the cables going through the channels as indicated in the diagram. Taking shortcuts around the channels is not recommended, especially if it leaves cables loose inside the suit, since this may put additional stress on the cables when the suit is worn.
• Make sure the cables are going the right way before you start feeding them through the channels. The ends with the labels (T, LA, RA, LL, RL) need to come out of the back right pocket to go into the hub.

We currently have five suit fabrics: two size Small, two size Medium, and one size Large. Note that it takes some time to move the sensors from one fabric to another (budget about an hour), so information about participants' sizing should be sorted out in advance of recording sessions.?

Choosing the correct suit size:?The suit should be nearly skin-tight. If it is too big for the wearer, the sensors will move around relative to the body and tracking will be poor.

Connecting the suit

1. The battery goes in the left pocket of the suit and the hub goes in the right. There is a hole between pockets for the connecting cable to pass through.
2. The sensor strings need to be plugged into the hub in the correct order: Ch1 T, Ch2 LA, Ch3 RA, Ch4 LL, Ch5 RL.
3. If connecting by Wi-Fi:
   • Connect the router to (i) the Nansense computer and (ii) the ethernet port on the wall. Wired Internet access is needed for the router to verify its license.
   • Open a Internet browser window on the computer and type the IP address of the hub (either 168.192.50.98 or 168.192.50.99). It may take a few minutes of refreshing the page to connect, but eventually the dashboard will load. If you are using the suit with gloves, the dashboard should look like this [insert image].?
4. If connecting by cable:
   • Open Control Panel/Network and Internet/Network and Sharing Center. Click Change Adapter Settings in the sidebar, then right-click the router and open properties. Select Internet Protocol Version 4 (TCP/IPv4), click properties, and enter the following:?computer static IP 192.168.157.200 and subnet mask 255.255.255.0
   • Connect the port on the hub to the computer with an ethernet cable.
   • Open an Internet browser window and enter 192.168.157.216.?The suit should connect quite quickly and the dashboard should appear.?
5. The dashboard will indicate when streaming has started. There will be a solid green light (labelled "live") on the hub at this point. You can then open the Nansense software and enable the button Toggle Live Stream. Type in the following and press the Connect System button:
   • If connecting by Wi-Fi: enter the IP address of the hub
   • If connecting by cable: enter?192.168.157.216
6. The avatar should appear in the display and a list of sensors should pop up on the right.
7. Note on connecting and disconnecting multiple hubs: It is possible to connect and stream multiple hubs simultaneously in the same software interface. Connect the hubs one at a time by entering each IP address and pressing Connect System. To disconnect from the hubs, press?Disconnect System. Note that the?Disconnect System?button only disconnects the most recent connection; therefore, hubs have to be disconnected in the reverse order in which they were connected.?

Calibration

In the viewport, put the visualization into magnetic mode to see if there is any magnetometer interference. It is important to minimize interference before calibrating.

It is best to use the full calibration procedure, if possible, which is initiated by pressing the?Calibration Wizard?button. This will take you through a series of actions to calibrate different parts of the suit. We normally set the time per step to 15 seconds.

If you need a fast calibration, ask the person wearing the suit to hold an I-pose or a T-pose as you press the Calibrate I-pose?or?T-pose?button. The I-pose is normally more effective than the T-pose.?

Re-calibration during recording is possible. Note, however, that running a new calibration erases all previous calibrations. This means that pressing?Calibrate I-pose?after running the multi-step?Calibration Wizard will erase the multi-step calibration.

Recording data

There are two ways to record: (i) data logging using the hub and (ii) directly in the software. We recommend using both methods simultaneously in the case of longer recordings (more than 14 minutes).

Recordings in the software are capped at 14 minutes duration for a 120 Hz sampling rate. When you reach the max length, the recording will stop automatically and your data will be exported. Longer recordings close to the max length can require up to about 2 minutes to export before another recording can start. To record in the software, simply press the record button, and press it again to stop the recording. Filename prefixes can be changed in the main menu (right side of the screen).

Recording in the hub using data logging?is necessary if you want to capture longer trials. Start data logging by pressing B2 on the hub. (There is currently no way to start data logging remotely.) The yellow "log" light on the hub will turn on, and "logging data" should pop up in the dashboard with a stopwatch showing the current recording length. It is recommended to start and stop data logs with an I-pose. This is important because data is initially logged without any calibration information, and calibration is carried out later during post-processing.?

Using Sync Box

We have a sync box which can be used to synchronize multiple suits and/or add LTC to the data. To use this you need the box to be on BEFORE the suit(s). To turn it on, simply connect it to the wifi router and plug in its power supply. If you are using LTC from Ambient or other device, use the connector to plug it into channel 1. The box should blink green when that is connected properly.?

When this is all connected then turn on the suit. On the Hub Dashboard for the chosen box, the sync box should show up as a green light.?

In the software, ensure Timecode is set to Hub/LTC inside the settings section.?

Accessing recorded data

When you record in the software, fbx (or bvh) and tc files are automatically saved to:?.\Nansensev20.2.0\dumps

When you record using data logging, tc files can be accessed through the dashboard. Go to?File Manager?to access and download saved logs. Downloaded logs should be moved to?.\Nansensev20.2.0\dumps\tc. Data logs are named in the format RECxxxx. There is no way to rename logs until they have been downloaded. There is also no information provided about when the log was created. It is therefore advisable to download a log and rename it as soon as possible.?It is also important to delete the log from the hub once it has been downloaded so as to avoid the hub becoming full of logs with no record of what they are or when they were created.

Playback of recorded data

Recordings can be played back in two ways, depending on the file format.

.fbx and .bvh files can be opened using the?Open File?button and played back using the transport buttons.

.tc files must be played back using the emulator. Open the?Emulator Menu, select the?file button, then select the .tc recording from the dropdown menu. Press?Create new. This creates a new IP address, which should be entered into the top toolbar (as though you are connecting a new hub). Enable Toggle Live Stream?and press?Connect System. Playback is controlled through the transport buttons in the Emulators toolbar. The recorded data will need to be calibrated at this point. Go to the point in the recording where the participant made an I-pose (ideally, you started and ended the recording with I-poses) and press?Calibrate I-pose.?To convert .tc files to .fbx, record the emulation by pressing the?Record?button, then starting the emulation playback. To remove the emulator, press the?Disconnect System?button and?Delete Emulator?button.

Skeleton Orientation

As Nansense is an inertial system, the global orientation of the skeleton is arbitrary. However, in the case that a specific rotational orientation is required, there are several things to note. First, in the Nansense software, the orientation of the positive direction of the z-axis is parsed as 0 degrees of rotational transform in exported fbx files. Second, the global orientation is set during the calibration process. Nansense provides two options: ¡°geomagnetic¡± and ¡°z axis¡±. These can be selected in the settings menu.

¡°Geomagnetic¡± orients the skeleton to face the geomagnetic north upon calibration. ¡°Z axis¡± orients the skeleton to face the positive z axis upon calibration. In exported fbx files, the ¡°Geomagnetic¡± option results in a rotation of approximately 127 degrees applied to the skeleton (I think this was correct, but need to check). ¡°Geomagnetic¡± results in a rotation of 0 degrees. It is not so simple to apply rotational transforms to the skeleton within the Nansense software, so if a specific orientation is required and the fbx file is not going to be worked on further within any software that allows rotational transforms to be applied, it is important to select the correct option before recording.

Applying a Rotational Transform

Applying a rotational transform to a capture requires several steps. First, editing must be enabled in the edit menu.

Second, ¡°system¡± must be selected in the selection dropdown menu.

Third, ¡°rotate select¡± must be selected in the editing toolbar.

The skeleton can then be clicked on and rotated. However, any rotation applied is not saved in an exported fbx file. Instead, the entire capture must be played through recorded once again with the rotational transform applied and saved as a new fbx file. For long or a large number of captures, this can be extremely time consuming. Therefore, it is recommended to perform any editing requiring rotation in separate animation software such as Blender.

Body Measurements

The landmarks used for the measurement of the body in the skeleton menu seem to vary. There is also little information in the documentation. For the captures with Helle, we firstly tried using the landmarks provided by Nansense when Laura contacted them early in the year. However, these provided a skeleton greatly out of proportion. After some experimentation we settled on the following landmarks:

JH: Extra measurement: shoulder width: the distance between the ends of the clavicles

These must be applied before the capture takes place, and importantly, are not applied until the ¡°generate body¡± button has been clicked.

Recording Stop

When recording multiple captures of the same subject, it is important to press the recording button to stop the recording instead of the stop button. If the stop button is pressed, all settings relating to the skeleton, such as the body measurements, will be reset and must be entered again.

Centering the Skeleton

To place the position of the skeleton on the origin on the horizontal plane, the ¡°center start displacement¡± button can be used. This positions the first frame of the capture upon the origin, so the order of operations must be considered if any trimming of the capture is necessary.

Trimming the Capture

The capture can be trimmed through either trimming all frames before or after the current frames by using the ¡°crop start¡± or ¡°crop end¡± buttons, or the corresponding shortcut buttons in the playback toolbar.

Note that this does not carry over any start frame translations applied to the skeleton, such as centering the start frame, so care should be taken with the order of operations.

By default, the Nansense software displays the skeleton in a t-pose in the first frame of the capture, not the actual skeleton data. This can make it more difficult to check if the trim has been applied correctly. In order to view the skeleton data of the first frame, a ¡°1¡± should be entered in the transport ¡°Frames¡± count. This will position the visualisation of the skeleton according to the data of the first frame.

?

TC vs Skeleton

Nansense provides captured data in two formats: .tc and either .fbx/.bvh. The .tc format is the raw sensor data and the .fbx/.bvh is the skeleton constructed from the raw sensor data. The ways of working with these two data types differ within the Nansense software, so care should always be taken to be aware of the data type being worked with.

The most important thing to note is that the .tc files can be played back within the software through the use of the emulator. The advantage of this is that multiple skeletons can be constructed from the same capture (for example, using different body measurements if an error is noted). However, this requires loading the .tc file into the emulator and then recording once more. This means that the entire capture must be played through in real-time. As this can take a long time for a large number of captures, this should be treated as a last resort and the aim should be to get as good a skeleton as possible constructed during the capture itself.

The .tc files can be viewed in a text editor. Each sensor consists of six data points. As these are unlabelled and there is little documentation, we are currently in the process of working out what these data points correspond to.

An additional point to note is that the .tc file adds an additional two ghost sensors when using the Pro_Suit_Gloves.ns model. These are added to the end of the foot, and double the data acquired from the single foot sensor with a lateral transform applied.

Real-Time Streaming

Nansense can also stream data in real-time. This is simple to set up. Within the broadcast menu, the ip address of the client can be set and the streaming format selected from the dropdown menu.

These are primarily various formats for the streaming of the skeleton data specifically formatted for specific animation softwares (Maya, Unity, etc.). However, the TC Raw option streams the raw skeleton data.

The data is streamed as a single byte string that contains all of the skeleton data/all sensor data for each frame. This needs to be decoded, formatted, and converted to the appropriate data type on the client end. Currently we are working on a Python script and Max patch which parses the received TC Raw stream into a usable format.

Care with Gloves

The gloves can break extremely easily, especially at the connector and if they are left to hang from the cable. It is very tempting for the subject wearing the suit to take off the gloves at the end of or between captures while they are still connected to the rest of the suit. It is very important that this does not happen and instead, the gloves should be disconnected from the rest of the suit before they are removed. This should be stated clearly at the start of each capture session.

Repairing Broken Connectors

In the event of a broken connector, it is possible to re-solder these in-house.

It is important to note that each repair removes approx. 1-3cm of length from the wire. This means that connectors with little slack in the wire (i.e., the gloves) can likely only be repaired 3-4 times each before the wire become unusably short. Minimising any chances for the connectors to break in the first place through careful use is always preferable!

Pin Diagram

The grey wire used in the sensor chains consists of four individually sheathed stranded wires: white, red, green, and black. Each wire MUST be connected to a specific pin on the connector when resoldering, as shown in the diagram below. The order is dependent on whether the broken connector is a male (i.e., visible pins) or female (i.e., holes to receive the pins) connector.

Connector Components

Each connector comes apart into three pieces, shown in the figure below:

• Piece 1: the main piece containing the pins/holes to which the wires must be soldered.
• Piece 2: a piece which threads onto Piece 1 and slides over the heatshrink in order to hold the wire in place.
• Piece 3: a threaded piece which threads onto Piece 2, and clamps Piece 2 into place over the heat shrink.

L-R: Pieces 1-3 of a male connector

The female connector looks similar to the above except for Piece 1, which features a protruding section to fit into the slot. Note the different ends in the figure below.

L: Male connector, R: Female connector

Soldering Instructions

These instructions are split into three sections: disassembling the broken connector, re-soldering the connector, and reassembling the fixed connector.

Required tools / components:

• Soldering iron
• Solder
• 3-5cm piece of thick heat shrink (from Kayla's office)
• Heat gun
• Craft knife
• Wire stripper
• Multimeter (recommended)
• Third hand / vice (recommended)
• Desoldering tool & desoldering wick (recommended)

Taking Apart the Broken Connector

1. Unscrew Piece 3 from Piece 2. This part should come loose and be able to be slid down the wire.
2. Unscrew Piece 1 from Piece 2 until Piece 1 comes loose.
   1. You may feel resistance if one or more of the wires is still connected to Piece 1. The action of unscrewing will break these connections - this is necessary as you will need to re-solder them anyway.
3. On Piece 1, you should see some remains of the wires and/or their solder. Carefully remove all the leftover solder and cable from the connector.
   1. Ideally, you should remove the solder from the holes on the back of the connector. This will give something for the cables to fit into when resoldering, significantly improving the strength of the solder.
   2. The plastic parts of the connectors melt very quickly when in contact with the soldering iron.
4. Pull Piece 2 off the exposed wire. This might take some force, as it clamps into the heat shrink.
5. Carefully remove the old heat shrink from the wire. You will likely need to cut it off using a?craft knife. As far as possible, do not cut into the grey shroud around the wire.

Re-Soldering the Connector

1. If necessary, strip back a small amount (max 10mm) of the grey shroud around the wire, to expose a small length of each of the coloured wires. This shroud contains metal fibers, so can take some effort to remove cleanly. A craft knife generally works better than a wire stripping tool.

2. Strip back a small amount (2-4mm) of the shroud from each of the coloured wires to expose some of the stranded cable. The wires are thin enough that the shroud can be removed using a wire stripping tool. These are the wires that you will solder into the connector.

3. Parts 2 & 3 and a small length of heat shrink MUST be on the wire before soldering the wires back onto Piece 1, as they will not fit over Piece 1 once it has been re-soldered to the wire. Place the pieces onto the wire in the following order: Piece 3, Piece 2, heat shrink.

4. Solder each coloured wire into the correct hole on the connector according to the pin diagram above. It can be helpful to hold Piece 1 in a third hand or clamp to hold it steady.

   • It can be worthwhile to check your solder connections using a multimeter. This is easier when repairing a male connector, as the multimeter can directly contact the pins of the connector.

Re-Assembling the Connector

1. Slide the heat shrink down so that it partially covers the newly soldered wires. Don't slide it all the way down as this prevents Pieces 2 and 3 from sliding over the top of it. Use a heat gun to shrink the heat shrink over the soldered wires.
2. Slide Piece 2 down over the heat shrink and screw it into Piece 1. This piece will likely rub against the heat shrink slightly, causing the wire to rotate. As far as possible, hold the wire and Piece 1 in place to prevent the wire from turning.?Turn Piece 2, not Piece 1, as turning Piece 1 WILL break the newly re-soldered connections!
3. Slide Piece 3 down the wire and screw it onto Piece 2. Piece 3 clamps the plastic pins on Piece 2 into the heat shrink to prevent the wire from slipping, therefore it can take some force to get Piece 3 over Piece 2.
4. Test your connection by re-assembling and setting up the suit.

Troubleshooting

Difficulties connecting the suit or starting streaming

Cannot connect to the suit via Wi-Fi

If you have recently used a suit in tethered mode, the network settings might be incorrect. Go to Control panel/Network and Internet/Network and Sharing Center/Change Adapter Settings. Click on the router and open properties. Select Internet Protocol Version 4 (TCP/IPv4)/Properties and check "Obtain an IP address automatically".

The dashboard does not show the correct number of sensors per chain

If you are positive that the chains are connected correctly (with head, feet, and gloves) and in the correct order (T, LA, RA, LL, RL in channels 1-5), then one of the chains may have a broken connection. Refer to Kayla for help.

It is not possible to remove the problematic chain and use the suit with the remaining sensors. Nansense treats the sensors as daisy-chained, so if the Ch 3 chain is broken, Ch 1 and Ch 2 will display properly, but the Ch 3-5 chains will not. It is also not possible to switch chains between body parts, even if the number of sensors is the same.?

However, if the configuration was used differently before (example, no gloves then gloves) you will have to run the autodetect procedure:

• Stop Steaming:?Tap B1 and wait a few seconds for the green LED to stop blinking.
• Autodetect:?Hold B1 pressed down. The green LED will blink rapidly for five seconds. The red LED will blink once the autodetect procedure has finished.
• Start Streaming:?Tap B1 and wait a few seconds. The green LED will blink slowly and after five seconds it will stay on, meaning that data is streaming

Dashboard alternates between "restarting streaming" and "streaming" with green light on the hub flashing rapidly and/or avatar does not appear when you try to connect to the suit in the Nansense software

Try restarting the streaming by doing the following using the buttons on the hub:

1. Push B1 one briefly and wait for the green light on the hub to turn off. This turns off streaming.
2. Push and hold B1 for several seconds until the green light stops blinking and the red light blinks once.?
3. Push B1 again briefly to restart streaming.

Poor data quality

Data has substantial drift

The Nansense suits start to drift very quickly, especially if there is some degree of magnetometer interference. You can monitor the degree of interference by viewing the avatar in "Magnetic" mode and/or checking the colour labels of individual sensors in the?Sensor Status?list at the bottom of the main menu (right toolbar). Red indicates strong interference, yellow indicates moderate interference, green indicates little or no interference.?

If a sensor is brought close enough to ferrous material that interference occurs, it must be brought a lot further away from the material than was required to create interference in order to return to a low level of interference. The sensors can also quite easily interfere with one another, especially the sensors located on the hands.

The avatar has its arms on backwards

Try inverting the compass sensor (under Settings/Calibration).

Poor data quality when running logged data through the emulator

This is a known issue that we are working with Nansense to resolve.