Continuing my personal journey into learning more about , I wanted to create a graph visualization of data stored in a graph database.  Graph visualization (or link analysis) presents data as a network of entities that are classified as nodes and links. To illustrate, consider this very simple network diagram: Dgraph Slash GraphQL While not a perfect example, one can understand the relationships between various services (nodes) and their inner-connectivity (links). This means the X service relies on the Y service to meet the needs of the business. However, what most may not realize is the additional dependency of the Z service, which is easily recognized by this illustration. For this article, I wanted to build a solution that can dynamically create a graph visualization. Taking this approach, I will be able to simply alter the input source to retrieve an entirely different set of graph data to process and analyze. The Approach Instead of mocking up data in a Spring Boot application (as noted in the " " article), I set a goal to utilize actual data for this article. Tracking the Worst Sci-Fi Movies With Angular and Slash GraphQL From my research, I concluded that the key to a building graph visualization is to have a data set that contains various relationships. Relationships that are not predictable and driven by uncontrolled sources. The first data source that came to mind was Twitter. After retrieving data using the Twitter API, the JSON-based data would be loaded into a Dgraph Slash GraphQL database using a somewhat simple Python program and a schema that represents the tweets and users captured by and uploaded into Slash GraphQL. Using the and the graph visualization library, the resulting data will be graphed to visually represent the nodes and links related to the #NeilPeart hashtag. The illustration below summarizes my approach: twarc Angular CLI ngx-graph Retrieving Data From Twitter Using `twarc` While I have maintained a semi-active Twitter account ( ) for almost nine years, I visited the to create a project called "Dgraph" and an application called "DgraphIntegration". This step was necessary in order to make API calls against the Twitter service. @johnjvester Twitter Developer Portal The solution (by DocNow) allows for Twitter data to be retrieved from the Twitter API and returned in an easy-to-use, line-oriented, JSON format. The twarc command line tool was written and designed to work with the Python programming language and is easily configured using the command and supplying the following credential values from the "DgraphIntegration" application: twarc twarc configure CONSUMER_KEY CONSUMER_SECRET ACCESS_TOKEN ACCESS_TOKEN_SECRET With the death of percussionist/lyricist , I performed a search for hashtags that continue to reference this wonderfully-departed soul. The following search command was utilized with twarc: Neil Peart twarc search #NeilPeart > tweets.jsonl Below is one example of the thousands of search results that were retrieved via the twarc search: { : , : , : { : , : }
} "content" "It’s been one year since he passed, but the music lives on... he also probably wouldn’t have liked what was going on in the word. Keep resting easy, Neil #NeilPeart https://t.co/pTidwTYsxG" "tweet_id" "1347203390560940035" "user" "screen_name" "Austin Miller" "handle" "AMiller1397" Preparing Dgraph Slash GraphQL Starting in September 2020, Dgraph has offered a fully managed backend service, called Slash GraphQL. Along with a hosted graph database instance, there is also a RESTful interface. This functionality, combined with 10,000 free credits for API use, provides the perfect target data store for the #NeilPeart data that I wish to graph. The first step was to create a new backend instance, which I called : tweet-graph Next, I created a simple schema for the data I wanted to graph: type User { : ! @id @search(by: [hash])
  screen_name: ! @search(by: [fulltext])
}

type Tweet { : ! @id @search(by: [hash])
  content: !
  user: User
}

type Configuration { : ID
  search_string: !
} handle String String tweet_id String String id String The and types house all of the data displayed in the JSON example above. The type will be used by the Angular client to display the search string utilized for the graph data. User Tweet Configuration Loading Data into Slash GraphQL Using Python Two Python programs will be utilized to process the JSON data extracted from Twitter using twarc: convert - processes the JSON data to identify any Twitter mentions to another user upload - prepares and performs the upload of JSON data into Slash GraphQL The core logic for this example lives in the upload program, which executes the following base code: data = {}
users = {}

gather_tweets_by_user()

search_string = os.getenv( )
print(search_string)

upload_to_slash(create_configuration_query(search_string)) handle data:
    print( )
    upload_to_slash(create_add_tweets_query(users[handle], data[handle])) 'TWARC_SEARCH_STRING' for in "=====" The organizes the Twitter data into the and objects. gather_tweets_by_user() data users The stores the search that was performed into Slash GraphQL for use by the Angular client upload_to_slash(create_configuration_query(search_string)) The loop processes the and objects, uploading each record into Slash GraphQL using for data user upload_to_slash(create_add_tweets_query(users[handle], data[handle])) Once the program finishes, you can execute the following query from the API Explorer in Slash GraphQL: query MyQuery {
  queryTweet {
    content
    tweet_id
    user {
      screen_name
      handle
    }
  }
}

query MyQuery {
  queryConfiguration {
    search_string
  }
} Using `ngx-graph` With Angular CLI The Angular CLI was used to create a simple Angular application. In fact, the base component will be expanded for use by , which was installed using the following command: ngx-graph npm install @swimlane/ngx-graph --save Here is the working AppModule for the application: @NgModule({ : [
    AppComponent
  ], : [
    BrowserModule,
    BrowserAnimationsModule,
    AppRoutingModule,
    HttpClientModule,
    NgxGraphModule,
    NgxChartsModule,
    NgxSpinnerModule
  ], : [CUSTOM_ELEMENTS_SCHEMA], : [], : [AppComponent]
}) { } declarations imports schemas providers bootstrap export class AppModule In order to access data from Slash GraphQL, the following method was added to the GraphQlService in Angular: allTweets:string = + + + + + + + + + ;

getTweetData() { .http.get<QueryResult>( .baseUrl + + .allTweets).pipe(
    tap(),
    catchError( { ErrorUtils.errorHandler(err)
    }));
} 'query MyQuery {' '  queryTweet {' '    content' '    tweet_id' '    user {' '      screen_name' '      handle' '    }' '  }' '}' return this this '?query=' this => err return Preparing Slash GraphQL to Work With `ngx-graph` The data in Slash GraphQL must be modified in order to work with the framework. As a result, a ConversionService was added to the Angular client, which performed the following tasks: ngx-graph createGraphPayload(queryResult: QueryResult): GraphPayload { graphPayload: GraphPayload = GraphPayload(); (queryResult) { (queryResult.data && queryResult.data.queryTweet) { tweetList: QueryTweet[] = queryResult.data.queryTweet;

      tweetList.forEach( { tweetNode:GraphNode = .getTweetNode(queryTweet, graphPayload); userNode:GraphNode = .getUserNode(queryTweet, graphPayload); (tweetNode && userNode) { graphEdge:GraphEdge = GraphEdge();
          graphEdge.id = ConversionService.createRandomId(); (tweetNode.label.substring( , ) === ) {
            graphEdge.label = ;
          } {
            graphEdge.label = ;
          }

          graphEdge.source = userNode.id;
          graphEdge.target = tweetNode.id;
          graphPayload.links.push(graphEdge);
        }
      });
    }
  } .log( , graphPayload); graphPayload;
} let new if if let ( ) => queryTweet let this let this if let new if 0 2 'RT' 'retweet' else 'tweet' console 'graphPayload' return The resulting structure contains the following object hierarchy: {
  links: GraphEdge[] = [];
  nodes: GraphNode[] = [];
} {
  id: string;
  label: string;
  source: string;
  target: string;
} {
  id: string;
  label: string;
  twitter_uri: string;
} export class GraphPayload export class GraphEdge implements Edge export class GraphNode implements Node While this work could have been completed as part of the load into Slash GraphQL, I wanted to keep the original source data in a format that could be used by other processes and not be proprietary to . ngx-graph Configuring the Angular View When the Angular client starts, the following method will fire, which will show a spinner while the data is processing. Then, it will display the graphical representation of the data once Slash GraphQL has provided the data and the ConversionService has finished processing the data: OnInit ngOnInit() { .spinner.show(); .graphQlService.getConfigurationData().subscribe( { (configs) { .filterValue = configs.data.queryConfiguration[ ].search_string; .graphQlService.getTweetData().subscribe( { (data) { queryResult: QueryResult = data; .graphPayload = .conversionService.createGraphPayload(queryResult); .fitGraph(); .showData = ;
        }
      }, (error) => { .error( , error);
      }).add( { .spinner.hide();
      });
    }
  }, (error) => { .error( , error);
  }).add( { .spinner.hide();
  });
} this this => configs if this 0 this => data if let this this this this true console 'error' => () this console 'error' => () this On the template side, the following tags were employed: ngx <ngx-graph *ngIf= = layout= [view]= [showMiniMap]= [zoomToFit$]= [links]= [nodes]= > <svg:marker id="arrow" viewBox="0 -5 10 10" refX="8" refY="0" markerWidth="4" markerHeight="4" orient="auto">
      <svg:path d="M0,-5L10,0L0,5" class="arrow-head" />
    </svg:marker>
  </ng-template>

  <ng-template #nodeTemplate let-node>
    <svg:g class="node" (click)="clickNode(node)">
      <svg:rect
        [attr.width]="node.dimension.width"
        [attr.height]="node.dimension.height"
        [attr.fill]="node.data.color"
      />
      <svg:text alignment-baseline="central" [attr.x]="10" [attr.y]="node.dimension.height / 2">
        {{node.label}}
      </svg:text>
    </svg:g>
  </ng-template>

  <ng-template #linkTemplate let-link>
    <svg:g class="edge">
      <svg:path class="line" stroke-width="2" marker-end="url(#arrow)"></svg:path>
      <svg:text class="edge-label" text-anchor="middle">
        <textPath
          class="text-path"
          [attr.href]="'#' + link.id"
          [style.dominant-baseline]="link.dominantBaseline"
          startOffset="50%"
        >
          {{link.label}}
        </textPath>
      </svg:text>
    </svg:g>
  </ng-template>

</ngx-graph> "showData" class "chart-container" "dagre" "[1720, 768]" "false" "zoomToFit$" "graphPayload.links" "graphPayload.nodes" < # > ng-template defsTemplate The tags not only provide a richer presentation of the data but also introduce the ability to click on a given node and see the original tweet in a new browser window. ng-template Running the Angular Client With the Angular client running, you can retrieve the data from the Slash GraphQL by navigating to the application. You will then see a  user experience similar to below: It is possible to zoom into this view and even rearrange the nodes to better comprehend the result set. Please note: For those who are not fond of the "dagre" layout, you can adjust the property to another in . ngx-graph.layout graph layout option ngx-graph When the end-user clicks a given node, the original message in Twitter displays in a new browser window: Conclusion A fully-functional Twitter Graph was created using the following frameworks and services: Twitter API and Developer Application twarc and custom Python code Dgraph Slash GraphQL Angular CLI and ngx-graph In a matter of steps, you can analyze Twitter search results graphically, which will likely expose links and nodes that are not apparent through any other data analysis efforts. This is similar to the network example in the introduction of this article that exposed a dependency on the Z service. If you are interested in the full source code for the Angular application, including the Python import programs referenced above, please visit the following repository on GitLab: https://gitlab.com/johnjvester/tweet-graph-client Have a really great day! Also published at https://dzone.com/articles/creating-a-twitter-graph-using-slash-graphql

The Graph

First Data

Target

Twitter

Creating A Meme Sharing App Using Slash GraphQL

Salesforce: Walled Garden No More

I ♥ writing code. Find me on GitLab.

Nominated for 2022 - HackerNoon Contributor of the Year - Javascript

Nominated for 2022 - Hackernoons Tech Storyteller of the Year

Nominated for 2022 - HackerNoon Contributor of the Year - Cloud Computing

Nominated for 2022 - HackerNoon Contributor of the Year - Heroku

Nominated for 2022 - HackerNoon Contributor of the Year - Design

Nominated for 2022 - HackerNoon Contributor of the Year - Salesforce

Too Long; Didn't Read

How To Make a Twitter Graph with Slash GraphQL

How To Make a Twitter Graph with Slash GraphQL

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

5 Signs You’ve Built a Secretly Bad Architecture (And How to Fix It)

Top 10 JavaScript Charting Libraries for Every Data Visualization Need

10 Data Table Libraries for JavaScript

10 Best React Native Chart Libraries

10 Best Datasets for Time Series Analysis

12 Mistakes that Data Scientists Make and How to Avoid Them

5 Signs You’ve Built a Secretly Bad Architecture (And How to Fix It)

Top 10 JavaScript Charting Libraries for Every Data Visualization Need

10 Data Table Libraries for JavaScript

10 Best React Native Chart Libraries

10 Best Datasets for Time Series Analysis

12 Mistakes that Data Scientists Make and How to Avoid Them

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps