|
VRPN 06.04 |
vrpn_Sound User Manual (version: 5/16/2000) This document is a user manual for vrpn_Sound. Functionality of vrpn_Sound_Client and vrpn_Sound_Server are described in detail in this document. vrpn_Sound is the base class for vrpn_Sound_Client and vrpn_Sound_Server. This document is broken into two main sections: client, server. This manual describes the underlying structure of the sound client and server. The descriptions of specific structures are only one interpretation of them. Users of these classes can use the structures to transport other information than which they were meant. For example, it is up to the programmer to always use radians or always use degrees. Although this document might mention one or the other, it is ultimately up to the programmer as to how the structures are interpreted. Be consistent. vrpn_Sound vrpn_Sound is used to provide some of the base VRPN functionality to classes that inherit from it. It is suggested that users start at a higher level than vrpn_Sound (at least vrpn_Sound_Client or vrpn_Sound_Server). vrpn_Sound is not discussed in this manual. vrpn_Sound_Client The sound client is derived publicly from vrpn_Sound and vrpn_Text_Receiver. vrpn_Sound provides the base VRPN functionality for the client. vrpn_Text_Receiver provides a mechanism for the client to receive messages from the vrpn_Sound_Server. This is useful for reporting warning and errors that might occur with the sound card or server in general. There are two possibly ways to use this class: instantiated or inherited. Inheriting from vrpn_Sound_Client makes it possible to override the receiveTextMessage function. The function simply prints out messages it gets from the server. All messages, whether text, warnings, or errors are treated in the same manner. By overriding this function it is possible to treat the messages differently, define error levels, etc. As with all VRPN code it is necessary to call the mainloop function periodically (maybe after each function call) to ensure that messages are passed around the network. Constructor/ destructor: vrpn_Sound_Client(const char * name, vrpn_Connection * c); ~vrpn_Sound_Client(); vrpn_Sound_Client takes a device name and a pointer to a vrpn_Connection. The device name is the name assigned in the vrpn.cfg file. Them following code returns a pointer to a sound client which is attached to the device "sound" on a connection to "oxygen-cs.cs.unc.edu" (note that we do a little busy waiting until the connection is set up): vrpn_Connection * connection; vrpn_Sound_Client * client; connection = new vrpn_Synchronized_Connection("oxygen-cs.cs.unc.edu"); mymanager = new vrpn_Sound_Client("sound", connection); while (!connection->connected()) {
} The destructor currently does nothing. Loading an audio scene: vrpn_int32 LoadModel_local(const char *filename); vrpn_int32 LoadModel_remote(const char *data); vrpn_int32 LoadPolyQuad(const vrpn_QuadDef quad); vrpn_int32 LoadPolyTri(const vrpn_TriDef tri); vrpn_int32 LoadMaterial(const vrpn_int32 id, const vrpn_MaterialDef material); The audio scene can be drawn by either loading a sound model file on the sound server machine, or by loading individual geometry. The provision to load a model from the sound client side is currently unavailable. LoadModel_local takes a filename which identifies the sound model file to be loaded. The format of this file is defined by the server code. See documentation on the individual servers for this information. The filename provided should be an absolute path to the file. The following line of code loads the model file found at C:\Aureal_Sound_Server\cube_model.dat on the sound server machine. Load_Model_local("C:\Aureal_Sound_Server\cube_model.dat"); Drawing geometry by using the LoadPoly* functions is tedious work, although it is about the same as writing model files from scratch by hand. These functions are used in conjunction with the LoadMaterial function. These functions use structures define in vrpn_Sound.h (vrpn_MaterialDef, vrpn_TriDef, vrpn_QuadDef). typedef struct _vrpn_MaterialDef {
} vrpn_MaterialDef; Material names should be descriptive of the material being modeled. The maximum length for a descriptive name is MAX_MATERIAL_NAME_LENGTH currently defined as 128 characters. The transmittance values describe how sounds move through the material while the reflectance values describe how sounds are reflected off of surfaces made of this material. Both of these have a gain and a high frequency value associated with them. The gain value determines how much the entire signal is attenuated, while the highfreq value determines how much the higher frequencies are attenuated. Some sample definitions might be (from Aureal’s A3D 3.0 API manual):
typedef struct _vrpn_QuadDef {
} vrpn_QuadDef; This structure is for defining quadrangles in the audio scene. The subQuad parameter acts as a boolean as to whether this quadrangle is acting as a subquad or not. Subquads are useful for defining windows or other openings in walls. The openingFactor parameter describes how "transparent" this quadrangle is. If the subQuad parameter is 0 then openingFactor is ignored. An open window would have an opening factor of 1.0, while shuttered windows might have an opening factor of 0.6. The tag parameter is a unique identifier for the surface. The material_name should be assigned to a previously defined material. typedef struct _vrpn_TriDef {
} vrpn_TriDef; This structure is similar to the QuadDef except it handles triangles instead of quadrangles. The following code defines a brick wall with a triangular shuttered window in the sound client (previously defined): vrpn_MatrialDef brickDef; vrpn_QuadDef wallDef; vrpn_TriDef windowDef; strcpy(brickDef.material_name, "BRICK"); brickDef.transmittance_gain = 0.95; brickDef.transmittance_highfreq = 0.60; brickDef.reflectance_gain = 0.40; brickDef.reflectance_highfreq = 0.20; // in this case the wall is parallel with the floor..that’s ok strcpy(wallDef.material_name,"BRICK"); wallDef.subQuad = 0; wallDef.openingFactor = 0.0; // doesn’t matter but set it anyway wallDef.tag = 1; wallDef.vertices[0][0] = 10.0; wallDef.vertices[0][1] = 10.0; wallDef.vertices[0][2] = 0.0; wallDef.vertices[1][0] = 10.0; wallDef.vertices[1][1] = -10.0; wallDef.vertices[1][2] = 0.0; wallDef.vertices[2][0] = -10.0; wallDef.vertices[2][1] = -10.0; wallDef.vertices[2][2] = 0.0; wallDef.vertices[3][0] = -10.0; wallDef.vertices[3][1] = 10.0; wallDef.vertices[3][2] = 0.0; strcpy(windowDef.material_name,"NONE"); windowDef.subQuad = 1; windowDef.openingFactor = 0.6; windowDef.tag = 1; windowDef.vertices[0][0] = 5.0; windowDef.vertices[0][1] = 0.0; windowDef.vertices[0][2] = 0.0; windowDef.vertices[1][0] = 0.0; windowDef.vertices[1][1] = -5.0; windowDef.vertices[1][2] = 0.0; windowDef.vertices[2][0] = 0.0; windowDef.vertices[2][1] = 5.0; windowDef.vertices[2][2] = 0.0; client->LoadMaterial(brickDef); client->LoadPolyQuad(wallDef); client->LoadPolyTri(windowDef); client->mainloop(); There are several functions defined that should allow previously defined materials and polygons to be altered (moved, made more transparent, materials decaying etc). At this time they are not supported and thus not described in this document. Sounds in the audio scene: vrpn_SoundID loadSound(const char* sound, const vrpn_SoundID id, const vrpn_SoundDef soundDef); vrpn_int32 unloadSound(const vrpn_SoundID id); vrpn_int32 playSound(const vrpn_SoundID id, vrpn_int32 repeat); vrpn_int32 stopSound(const vrpn_SoundID id); vrpn_int32 setSoundVolume(const vrpn_SoundID id, const vrpn_float64 volume); vrpn_int32 setSoundPose(const vrpn_SoundID id, vrpn_float64 position[3], vrpn_float64 orientation[4]); vrpn_int32 setSoundVelocity(const vrpn_SoundID id, const vrpn_float64 velocity[4]); vrpn_int32 setSoundDistances(const vrpn_SoundID id, const vrpn_float64 max_front_dist, const vrpn_float64 min_front_dist, const vrpn_float64 max_back_dist, const vrpn_float64 min_back_dist); vrpn_int32 setSoundConeInfo(const vrpn_SoundID id, const vrpn_float64 inner_angle, const vrpn_float64 outer_angle, const vrpn_float64 gain); vrpn_int32 setSoundDopScale(const vrpn_SoundID id, vrpn_float64 dopfactor); vrpn_int32 setSoundEqValue(const vrpn_SoundID id, vrpn_float64 eq_value); vrpn_int32 setSoundPitch(const vrpn_SoundID id, vrpn_float64 pitch); There are a number of functions for adding and manipulating sounds in the audio scene. Once sounds have been added to the scene, a specific sounds’ parameters can be changed. The unique sound identifier provided in the loadSound function is used to specify a particular sound in later functions. loadSound takes a filename for the sound, the unique identifier and a vrpn_SoundDef. vrpn_SoundDef is a structure that holds all information for a particular sound. The vrpn_PoseDef structure holds location information. typedef struct _vrpn_PoseDef {
} vrpn_PoseDef; position is a vector in XYZ order. Specification regarding the coordinate system is left up to the end users (client and server). orientation is a quaternion in XYZW order. quatlib is used to specify quaternion definitions. This pose structure is used for positioning sounds as well as listeners. The main point here is that the coordinates should be world coordinates! That is, where is the position relative to 0,0,0 in world coordinates and what is the orientation relative to no rotation in world coordinates. typedef struct _vrpn_SoundDef {
} vrpn_SoundDef; vrpn_SoundDef used vrpn_PoseDef to hold the position information. The velocity parameter is a vector of length four (which doesn’t make sense to me.. since we are simply defining velocity in the X, Y and Z directions [one idea is to maybe just set the first three values and ignore the fourth value]). The distance information is one of the more confusing aspects of the sound structure. The distance information describes how far away from the source the noise becomes inaudible (max_front_dist) and how close to source the maximum volume level is heard (min_front_dist). For example, setting max_front_dist to 15 and min_front_dist to five, means that further than 15 units from the source the sound can not be heard and anywhere closer than 5 units from the source the volume level is volume. min/max_back_dist is for when the listener has his/her back to the source. Usually this is the same as the front distances. The sound cone information specifies the sound cone. (The following is from the A3D 3.0 manual and is specific to the sound server based on the A3D libraries, but gives an idea how these parameters can be used.) "The two angles, [inner_angle] and [outer_angle] define the size of the cone. Between 0 [radians] and [inner_angle], the source will be at the level specified [by volume] (plus any effect caused by distance or occlusions). Between [inner_angle] and [outer_angle] the source gain is multiplied by the cone gain calculatyed by interpolating between 1.0 and [gain] according to the bearing of the listener from the source. From [outer_angle] to [PI radians], the source gain is multiplied by [gain]. Enabling a sound cone for a source, results in a small performance overhead as some extra calculatings need to be performed on the source. Setting either [outer_angle] to 0 or [gain] to 1 disabled cone processing and the sourc eis treated as omnidirectional." Make sure the client and the server use radians or degrees! The dopler_scale parameter allows the dopler shift on a source to be exagerated or reduced in the following manner: a value of 1.0 is the default setting, a value of 2.0 would double the effect while 0.5 would halve the effect. The equalization_val parameter is similar to a treble control on a stereo. The pitch parameter changes the playback rate of a sample. The volume parameter is actually the gain setting, which specifies the loudness of the source. The following code adds a source to the environment and begins playing the source: vrpn_SoundDef soundDef; for (int i=0; i<3;i++) soundDef[0].pose.position[i] = 0.0; for (i=0;i<3;i++) soundDef[0].pose.orientation[i] = 0.0; soundDef[0].pose.orientation[3] = 1.0; for (i=0;i<4;i++) soundDef[0].velocity[i] = 0.0; soundDef[0].max_front_dist = 5.0; soundDef[0].min_front_dist = 0.1; soundDef[0].max_back_dist = 0.1; soundDef[0].min_back_dist = 0.1; soundDef[0].cone_inner_angle = 0; soundDef[0].cone_outer_angle = 0; soundDef[0].cone_gain = 1; soundDef[0].volume = .8; soundDef[0].dopler_scale =1.0; soundDef[0].equalization_val = 1.0; soundDef[0].pitch = 1.0; client->loadSound("c:\\dance.wav", 0 , soundDef); client->mainloop(); In the above example we have loaded a sound into the audio scene. This sound can be manipulated with the setSound* functions. The identifier expected by these functions is the unique identifier supplied in the loadSound functions. These functions are fairly straight-forward and will not be described in detail other than to say that the values can be set while the source is playing or not. The playSound function takes a unique identifier and a repeat parameter. This parameter specifies the number of times the source should be played. A value of 0 indicates that the source should be played continously. It is wise to set the listener’s position before beginning to play sounds. Neglecting to do this could cause the source to appear to be playing from arbitrary locations. The stopSound and unloadSound functions are straight-forward. The Listener vrpn_int32 setListenerPose(const vrpn_float64 position[3], const vrpn_float64 orientation[4]); vrpn_int32 setListenerVelocity(const vrpn_float64 velocity[4]); The listener’s location is specified using the setListenerPose function. Curiously, it splits the pose into a position and orientation in the parameter list. This allows one or the other to be changed with out manipulating any underlying structures. Again, the orientation is a quaternion. The listener’s velocity can be set using the setListenerVelocity function. Why this function takes a vector of length four is a mystery. Other things: void mainloop(const struct timeval * timeout=NULL); virtual void receiveTextMessage(const char * message, vrpn_uint32 type, vrpn_uint32 level, struct timeval msg_time); As mentioned above the client’s mainloop function should be called to envoke VRPN message passing. It is recommended that the mainloop function be called after every vrpn_Sound_Client call if only so the programmer does not come to wonder why nothing is happening in the code to realize that the mainloop function was never called. The receiveTextMessage function is used to capture messages sent from the server back to the client. This is extremely helpful for transmitting error and warning messages. The vrpn_Sound_Client class is inherited from vrpn_TextReceiver which allows for this capability. The function is virtual and can be overriden to perform filtering or other functions. The default function simply prints out all messages from the server directly to stdout. vrpn_Sound_Server vrpn_Sound_Server is derived from the vrpn_Sound class (and vrpn_Text_Sender class) and must be inheritted from to be used. It provides an end point opposite the client code for sound. Classes built on top of this class will most likely be specific to a particular sound library. The rest of this document describes how to go about constructing the sound server. As VRPN messages are sent across the network to the sound server they are received and interpreted by vrpn_Sound_Client. Specific callbacks are called in response to these messages; it is the responsibility of the programmer to define the body of these callbacks. Below the functions are listed with a short description of what is being sent over and what should be done with that data. void playSound(vrpn_SoundID id, vrpn_int32 repeat, vrpn_SoundDef soundDef); When the client sends a play message to the server this function is called. The identifier should be a unique number that was specified in a loadSound* function. The repeat parameter specifies how many times the sample will play. A value of 0 indicates that the sample should play continuously. soundDef is actually not filled in, so any value within it should be ignored! void loadSoundLocal(char* filename, vrpn_SoundID id, vrpn_SoundDef soundDef); When the client sends a request to load a sound this function is called. The filename should be an absolute path to the sound file. There is no guarantee that the path or filename are correct, however. The identifier should be unique, so it is acceptable to overwrite any pre-existing sounds with this identifier. The soundDef should have all the parameters for the sound source. The following information is from the vrpn_Sound_Client portion of this document. The programmer can assume that is what will be sent over by the client, and this is what is meant be the values in the structure. typedef struct _vrpn_PoseDef {
} vrpn_PoseDef; position is a vector in XYZ order. Specification regarding the coordinate system is left up to the end users (client and server). orientation is a quaternion in XYZW order. quatlib is used to specify quaternion definitions. This pose structure is used for positioning sounds as well as listeners. The main point here is that the coordinates should be world coordinates! That is, where is the position relative to 0,0,0 in world coordinates and what is the orientation relative to no rotation in world coordinates. typedef struct _vrpn_SoundDef {
} vrpn_SoundDef; vrpn_SoundDef used vrpn_PoseDef to hold the position information. The velocity parameter is a vector of length four (which doesn’t make sense to me.. since we are simply defining velocity in X, Y and Z). The distance information is one of the more confusing aspects of the sound structure. The distance information describes how far away from the source the noise disappears (max_front_dist) and how close to source the maximum volume level is heard (min_front_dist). For example, setting max_front_dist to 15 and min_front_dist to five, means that further than 15 units from the source the sound can not be heard and anywhere closer than 5 units from the source the volume level is volume. min/max_back_dist is for when the listener has his/her back to the source. Usually this is the same as the front distances. The sound cone information specifies the sound cone. (The following is from the A3D 3.0 manual and is specific to the sound server based on the A3D libraries, but gives an idea how these parameters can be used.) "The two angles, [inner_angle] and [outer_angle] define the size of the cone. Between 0 [radians] and [inner_angle], the source will be at the level specified [by volume] (plus any effect caused by distance or occlusions). Between [inner_angle] and [outer_angle] the source gain is multiplied by the cone gain calculatyed by interpolating between 1.0 and [gain] according to the bearing of the listener from the source. From [outer_angle] to [PI radians], the source gain is multiplied by [gain]. Enabling a sound cone for a source, results in a small performance overhead as some extra calculatings need to be performed on the source. Setting either [outer_angle] to 0 or [gain] to 1 disabled cone processing and the sourc eis treated as omnidirectional." Make sure the client and the server use radians or degrees! The dopler_scale parameter allows the dopler shift on a source to be exagerated or reduced in the following manner: a value of 1.0 is the default setting, a value of 2.0 would double the effect while 0.5 would halve the effect. The equalization_val parameter is similar to a treble control on a stereo. The pitch parameter changes the playback rate of a sample. The volume parameter is actually the gain setting, which specifies the loudness of the source. void loadSoundRemote(char* file, vrpn_SoundID id, vrpn_SoundDef soundDef); This function is currently not supported and should not be called. One idea for when this does get implemented is that the actual sound file could be streamed over the network. void stopSound(vrpn_SoundID id); Stop the sound with this identifier. If the sound with this identifier does not exist then maybe a warning message should be sent back to the client. If the sound is not playing then maybe nothing should be done. void unloadSound(vrpn_SoundID id); Unload the sound with this identifier. Hopefully this sound has been loaded, if not a warning message should be sent back to the client. Once unloaded then the all traces of this sound source should be gone. It should be possible to reuse this identifier after this function has been called. Any memory taken up by this source should be flushed. void changeSoundStatus(vrpn_SoundID id, vrpn_SoundDef soundDef); If more than one of this sound’s parameters needs to change then this function is useful. See the information on vrpn_SoundDef in this section above. void setListenerPose(vrpn_PoseDef pose); See the information on vrpn_PoseDef above. The main thing to note is that this information is in world coordinates. Right now we only support one listener so there is no need to specify one. void setListenerVelocity(vrpn_float64 *velocity); A vector of length four is sent over. Why four? Not sure. The suggestion using the first three numbers as velocity in the X/Y/Z directions (respectively), and ignoring the fourth number. void setSoundPose(vrpn_SoundID id, vrpn_PoseDef pose); Note that the pose information should be in world coordinates. void setSoundVelocity(vrpn_SoundID id, vrpn_float64 *velocity); A vector of length four is sent over. Why four? Not sure. The suggestion using the first three numbers as velocity in the X/Y/Z directions (respectively), and ignoring the fourth number. void setSoundDistInfo(vrpn_SoundID id, vrpn_float64 *distinfo); This is a vector of length four. The distance information is in the following order inside this vector: min_back, max_back, min_front, max_front. See the description above within the vrpn_SoundDef discussion for details on what these mean. void setSoundConeInfo(vrpn_SoundID id, vrpn_float64 *coneinfo); This is a vector of length three. The cone information is in the following order inside this vector: inner_angle, outer_angle, gain. The vrpn_SoundDef description above has one interpretation of these numbers. void setSoundDoplerFactor(vrpn_SoundID id, vrpn_float64 doplerfactor); void setSoundEqValue(vrpn_SoundID id, vrpn_float64 eqvalue); void setSoundPitch(vrpn_SoundID id, vrpn_float64 pitch); void setSoundVolume(vrpn_SoundID id, vrpn_float64 volume); See the descriptions in the discussion on vrpn_SoundDef above to find some details on how these values can be interpreted. Note that volume is really gain (or the loudness of a source). void loadModelLocal(const char * filename); This function is called with a model file name. This file may or may not exist, the function should be able to handle both situations. The model file format is up to the programmer. void loadModelRemote(); This function is not supported. I can think of two ways to implement this function (the function of sending model data from the client side over to the server): encode the textual description of the model from the client-side and interpret it on the server-side, or read the model file on the client-side and call LoadMaterial and LoadPoly* functions from there. void loadPolyQuad(vrpn_QuadDef * quad); We receive a pointer to a vrpn_QuadDef. See the description in vrpn_Sound_Client about this structure. One question is what to do if the material specified has not been defined previously. void loadPolyTri(vrpn_TriDef * tri); We receive a pointer to a vrpn_QuadDef. See the description in vrpn_Sound_Client about this structure. One question is what to do if the material specified has not been defined previously. void loadMaterial(vrpn_MaterialDef * material, vrpn_int32 id); We get a unique identifier and a pointer to a vrpn_MaterialDef. This identifier should be unique, if not a warning message should be sent back to the client. void setPolyQuadVertices(vrpn_float64 vertices[4][3], const vrpn_int32 id); This function is not supported. The idea is to be able to move the vertices (dynamic objects?) once a polygon has been loaded. The implementation of this is non-trivial. void setPolyTriVertices(vrpn_float64 vertices[3][3], const vrpn_int32 id); This function is not supported. The idea is to be able to move the vertices (dynamic objects?) once a polygon has been loaded. The implementation of this is non-trivial. void setPolyOF(vrpn_float64 OF, vrpn_int32 tag); This function is not supported. The idea is to be able to change the opening factor once a polygon has been loaded. The implementation of this is non-trivial. void setPolyMaterial(const char * material, vrpn_int32 tag); This function is not supported. The idea is to change the material of an object once a polygon has been loaded. The implementation of this is non-trivial. Other Issues:
|