For this guide, we will be using to go through various concepts of real-time CRUD. For part 2, we will only consider the back end of our application. For the front end guide, see . https://github.com/SocketCluster/sc-crud-sample sc-crud-sample Real-time CRUD guide: Front end (part 1) As mentioned in part 1, to implement real-time CRUD on the back end, we need the following things: A database (in this case ). RethinkDB A WebSocket server + client framework which supports pub/sub ( ). SocketCluster A back end module to process CRUD requests and publish change notifications ( ). sc-crud-rethink The module is a plugin for SocketCluster; it should be initialized with a custom object as argument — this object contains a which defines the database entities that are supported by our application and the views through which they can be aggregated. sc-crud-rethink crudOptions schema The object is defined inside and is passed to the plugin via the method. Our file looks like this: crudOptions worker.js sc-crud-rethink attach worker.js ^ The object contains options that are needed to initialize the plugin. It may have the following properties: crudOptions sc-crud-rethink : An object which declares all the entity types and the views that are used by our application. schema [optional]: An object to use to log errors and warnings; defaults to . logger console [optional]: An object to pass to the RethinkDB module when instantiating it. thinkyOptions thinky [optional]: The default number of items per page for all views in the system; defaults to 10. defaultPageSize [optional]: A boolean which indicates whether or not to disable caching; defaults to false (cache enabled). cacheDisabled [optional]: The number of milliseconds to cache resources for; defaults to 10000 (10 seconds). Note that although cache updates/invalidates in real-time, it’s a good idea to keep this value short in order to save memory. cacheDuration [optional]: This is a boolean value which relates to access control ( middleware filter). It is false by default. If set to true, this option adds an extra layer of security by disabling all CRUD operations unless the relevant middleware explicitly allows them. (Access control filters will be explained in more detail later in this guide). blockPreByDefault pre pre [optional]: This boolean value has the same purpose as except that it applies to the middleware filter instead of . It is false by default. blockPostByDefault blockPreByDefault post pre Schema The most important part of the object is the object; the top level properties of this object represent the different resource types that are supported throughout the system (I.e. , and ); the name of each resource type represents a table name in our RethinkDB database. crudOptions schema Category Product User The resource type schema looks like this: Category Category: {fields: {id: type.string(),name: type.string(),desc: type.string().optional()},views: {alphabeticalView: {affectingFields: ['name'],transform: function(fullTableQuery, r) {return fullTableQuery.orderBy(r.asc('name'));}}},filters: {pre: mustBeLoggedIn}} ^ Each resource type definition contains a property which defines the fields that can be read from and modified on the underlying resource (along with type information for each field). The object under each resource type defines all the different ways in which the underlying resource can be aggregated into lists/collections. The object allows us to define custom middleware functions to block CRUD operations initiated by various sockets; this is useful mostly for access control (authorization) purposes. fields views filters The object is declared directly under the relevant resource type. For the resource type, the definition looks like this: fields Product fields fields: {id: type.string(),name: type.string(),qty: type.number().integer().optional(),price: type.number().optional(),desc: type.string().optional(),category: type.string()}, ^ This provides a definition of all the fields which can exist on the resource. The field should always be provided and must be of type . Note that some fields can be marked as optional. The plugin uses for data modeling; so the schema definition under the property above corresponds exactly to the schema definition for a model in Thinky. id string sc-crud-rethink Thinky fields The object is declared directly under the relevant resource type. For the resource type, the definition looks like this: views Product views views: {categoryView: {// Declare the fields from the Product model which// are required by the transform function.paramFields: ['category'],affectingFields: ['name'],transform: function(fullTableQuery, r, productFields) {// Because we declared the category field above, it is// available in here. This allows us to tranform/filter// the Product collection based on a specific category// ID provided by the frontend.return fullTableQuery.filter(r.row('category').eq(productFields.category)).orderBy(r.asc('name'));}},lowStockView: {// Declare the fields from the Product model which are required// by the transform function.paramFields: ['category', 'qty'],primaryKeys: ['category'],transform: function(fullTableQuery, r, productFields) {// Because we declared the category field above, it is// available in here.// This allows us to tranform/filter the Product collection// based on a specific category ID provided by the frontend.return fullTableQuery.filter(r.row('category').eq(productFields.category)).filter(r.row('qty').le(productFields.qty)).orderBy(r.asc('qty'));}}} ^ Here there are two views — a and a ; the purpose of a view is to transform an entire collection of resources from the database into a useful subset which can be consumed by components on the front end (and which can be updated in real-time). categoryView lowStockView SCCollection The most important aspect of each view is the function; the purpose of this function is to construct a RethinkDB ReQL query which will be used by the plugin to generate the view on demand. In the case of the resource type, the function’s argument is a ReQL query which, if executed directly, would produce a list of all items in the table. For the , the goal of the function is to transform the full table of resources into a smaller subset which only contains resources that belong to a specific id — the exact id to use depends on the argument (provided by a front end ). The argument which is passed to the transform function can be used to construct ReQL sub-queries and predicates. Note that the transform function doesn’t need to specify lower or upper bounds (limits) on the result set; this will be handled automatically by the pagination logic of . transform sc-crud-rethink Product transform fullTableQuery Product categoryView transform Product Product category category productFields.category SCCollection r sc-crud-rethink Each view declaration can have a property; this is an array of field names which will affect the view’s transformation depending on values provided by front end components (provided through a object). Using , a single view definition such as can be used to represent an infinite number of parameterized views in the form ; where can be replaced with any valid category id — front end components will decide what should be in each case. paramFields SCCollection viewParams paramFields categoryView categoryView({category: x}) x SCCollection x The property is similar to in that it is also an array of field names which affect the view’s transformation; the main difference is that fields listed under cannot be passed as parameters to the view by front end components. If we consider the transform function for the , we can see that the field affects the ordering of resources within the view but (unlike with the field) the field is only being used in a generic way: affectingFields paramFields affectingFields SCCollection categoryView name category name fullTableQuery.filter(r.row('category').eq(productFields.category)).orderBy(r.asc('name')); Under the definition, we defined a array — this is necessary because one of our values ( ) is used as part of an inequality predicate in our ReQL query: . As mentioned in part 1 of this guide, fields which are used for inequalities cannot be used as identifiers for the view. So in this case we have to be explicit about what the view’s primary keys are to make sure that they do not include the field. lowStockView primaryKeys paramFields productFields.qty r.row('qty').le(productFields.qty) qty If a view declares a list of explicitly on the back end, then components on the front end will need to provide a matching list of field names as when binding to that view. If a collection doesn’t update in real-time as expected when data changes, it could be because primary key declarations are missing on the front end and/or on the back end view schema. Important note: primaryKeys SCCollection viewPrimaryKeys SCCollection As mentioned earlier, each resource type in the schema definition can have a object which can be used to perform access control related to individual CRUD operations (initiated by logic on the front end). The object can have a and/or a property — in both cases, these properties should hold middleware functions in the form ; the object is an object with the following properties: filters filters pre post function (req, next) req (RethinkDB object which can be used to execute database queries) r r (SocketCluster socket which initiated the CRUD operation) socket (‘create’, ‘read’, ‘update’ or ‘delete’) action (a valid CRUD query should always have a property, it may also have various combinations of , , or properties depending on the specific kind of CRUD operation that is being performed — this is because some operations relate to an individual resource or an individual field within a resource while others relate to a view/collection). query type id field view viewParams The argument passed to the middleware function is a callback; we can invoke this callback with an error object (to block the CRUD operation) or without arguments (to allow the CRUD operation to proceed). next The difference between and filters is that filters do not need to query the underlying resource on the database before they run; on the other hand, filters will fetch the underlying resource from the database before they run (which adds extra overhead). Note that in the case of the filter function, the object has an additional property which holds the underlying resource from the database. pre post pre post post req resource In the schema, under the resource type, both a and a filter are specified: Product pre post filters: {pre: mustBeLoggedIn,post: postFilter} The filter function looks like this: mustBeLoggedIn function mustBeLoggedIn(req, next) {if (req.socket.authToken) {next();} else {next(true);req.socket.emit('logout');}} ^ It checks if the socket has a valid JWT ; if so, it will allow the CRUD action to proceed, otherwise it will block the CRUD action and will trigger a event on the client socket (to remove any stale tokens and to prompt the user to login). socket.authToken logout The filter on our resource type currently does nothing but it can be extended to perform advanced access control (as the comments suggest): post Product function postFilter(req, next) {// The post access control filters have access to the// resource object from the DB.// In case of read actions, you can even modify the// resource's properties before it gets sent back to the user.next();} ^ The filter works the same as the filter except that the object has an additional property which contains the resource from the database that will be affected by the CRUD operation. post pre req resource It’s more efficient to block CRUD operations using the filter, but there may be cases when we need to check the value of the underlying resource before we can determine whether or not to block the CRUD operation on that resource; so in that case, the filter is necessary. pre post Authentication The specifies most of the functionality of our CRUD application in a declarative way. The main part which hasn’t yet been covered is authentication (not to be confused with authorization/access control). For this, we have this line in : schema worker.js authentication.attach(scServer, socket); The logic behind this line is located in : sc_modules/authentication.js ^ In here we’re attaching a handler to a custom event: login socket.on('login', validateLoginDetails); This handler loads user details from the database and checks if the password provided by the front end socket matches the one stored in the database; if so, it will authenticate the socket by setting an on it using: authToken socket.setAuthToken(token, {expiresIn: tokenExpiresInSeconds}); ^ This function will attach a valid to both the server socket and the client socket. authToken If the provided password does not match the one in the database, we respond to the event with an error message . login 'Invalid username or password' Note that in this sample app, we store the password in the database as plaintext, however, in a real prodution app this would be bad security practice; we should only be storing a hash of the password instead (with salt) —so on , instead of comparing actual passwords, we should only be comparing the hashes. login Performing CRUD operations on the back end This functionality isn’t covered by the application but sometimes it may be necessary to update some data directly from the back end (not through a front end or component). For these scenarios, it’s possible to invoke CRUD operations directly from the back end using the object. For example: sc-crud-sample SCModel SCCollection crud var crud = scCrudRethink.attach(this, crudOptions); // Create a Product resource.crud.create({type: 'Product', value: {name: 'My product', ... }}, callback); // Read a Product resource's name field.crud.read({type: 'Product', id: 'b0658406-f0d8-4413-84e5-63157090bfcb', field: 'name'}, callback); // Update a Product resource's name field.crud.update({type: 'Product', id: 'b0658406-f0d8-4413-84e5-63157090bfcb', field: 'name', value: 'A good product'}, callback); // Delete a resource.crud.delete({type: 'Product', id: 'b0658406-f0d8-4413-84e5-63157090bfcb'}, callback); ^ Peforming CRUD operations like this on the back end will automatically trigger update notifications for all and instances on the front end; no extra logic required. SCModel SCCollection Manually triggering update notifications Sometimes we want to update some value in the database directly (without using the object). In these cases, we will need to trigger the update notification manually (so that affected views will still be updated in real-time). This can be achieved by calling the method: crud notifyUpdate crud.notifyUpdate({type: 'Product',oldResource: { ... },newResource: { ... }}); ^ The property should hold an object which represents the old resource (before the change took effect). The should hold an object which represents the new resource (after the update took effect). This function will trigger an update on both and components on the front end (for all affected views). oldResource newResource SCModel SCCollection
