Asking for much needed help to import a temporal multiplex multilayer graphml file into Netlogo and create an appropriate model setup

[pandora]

Hello, I am having a heck of a time trying to import a graphml file into netlogo and set up the model the way I need it. I’ve gone through all the network tutorials and I think the base problem is I don’t understand how to go from the graphml file with edge and node attributes to the netlogo vocabulary.
I made the graphml file in python with networkx-temporal from an edgelist and node list. I can get the nodes in as turtles, although they will be stationary and not move. But each edge is actually a vehicle that is moving, and each vehicle has multiple edges between the nodes over time. These are what is actually moving and what I need to follow in the model.

<?xml version='1.0' encoding='utf-8'?>
<graphml xmlns="http://graphml.graphdrawing.org/xmlns" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://graphml.graphdrawing.org/xmlns http://graphml.graphdrawing.org/xmlns/1.0/graphml.xsd">
  <key id="d8" for="edge" attr.name="weight" attr.type="long" />
  <key id="d7" for="edge" attr.name="breed" attr.type="string" />
  <key id="d6" for="edge" attr.name="vehID" attr.type="string" />
  <key id="d5" for="edge" attr.name="time" attr.type="long" />
  <key id="d4" for="node" attr.name="lat" attr.type="double" />
  <key id="d3" for="node" attr.name="long" attr.type="double" />
  <key id="d2" for="node" attr.name="prop_type" attr.type="string" />
  <key id="d1" for="node" attr.name="prop_grp_ID" attr.type="string" />
  <key id="d0" for="node" attr.name="propertyID" attr.type="string" />
  <graph edgedefault="directed">
    <node id="1">
      <data key="d0">P000174</data>
      <data key="d1">PG000001</data>
      <data key="d2">sows</data>
      <data key="d3">-82.25558967009326</data>
      <data key="d4">-32.27602112939915</data>
    </node>
    <node id="2">
      <data key="d0">P000179</data>
      <data key="d1">PG000001</data>
      <data key="d2">others</data>
      <data key="d3">-79.13847257689632</data>
      <data key="d4">-35.94708963637872</data>
    </node>
    <node id="49">
      <data key="d0">P000224</data>
      <data key="d1">PG000001</data>
      <data key="d2">sows</data>
      <data key="d3">-81.39624050937776</data>
      <data key="d4">-33.60766720710796</data>
    </node>
...
 <edge source="389" target="284" id="257549">
      <data key="d5">19318</data>
      <data key="d6">GPS000019</data>
      <data key="d7">animals</data>
      <data key="d8">1</data>
    </edge>
    <edge source="389" target="284" id="257649">
      <data key="d5">19363</data>
      <data key="d6">GPS000019</data>
      <data key="d7">animals</data>
      <data key="d8">1</data>
    </edge>
    <edge source="389" target="284" id="257970">
      <data key="d5">19474</data>
      <data key="d6">GPS000019</data>
      <data key="d7">animals</data>
      <data key="d8">1</data>
    </edge>

I am trying to import it into NetLogo and am having a heck of a time. I am new to NetLogo- trying to use it for my thesis.
The two layers of edges are the two veh_types edge attribute.
Each edgeID attribute is a different vehicle that i need to track along the edges of the network.
The end_time attribute is the discrete time of the temporal graph.

My thought was to import the nodes as one breed of turtle (that will stay stationary), and import the edgeID attribute (the vehicle name) as another breed of turtle since these will be moving. then use all the edges as ways the vehicles can move across. but the edgeID is an edge attribute in my edgelist.

I have tried AI help, read all the nw:extension documentation, roman-road network tutorial, looked through all the github models and examples, etc. I am all out of ideas.

the following code is the only one that has given me a network in the environment. The problem is that all the edges are just links and all the nodes are the turtles, but they aren’t supposed to move. I need the vehicles to move along the edges.

extensions [nw gis]
directed-link-breed [ edges edge ]
edges-own [ edgeID veh_type weight end_time]
turtles-own [ propertyID prop_grp_ID prop_type long lat ]  
links-own []
globals [ total-time current-time ]

;;;;;;;;;;;;;;;;;;;;
;;;;SETUP;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;

to setup
  clear-all
  
  gis:set-transformation [-180 180 -90 90 ] [-10 10 -5 5 ]
  
  nw:set-context turtles edges
  nw:load-graphml "practice_graph.graphml" 
  
  ask turtles [ setxy long lat ]
  
  set total-time 19843
  set current-time 19037
  
  reset-ticks

end

I’ve tried something like this and different varieties

extensions [nw gis]

; the different breeds for the edges
directed-link-breed [ veh_types veh_type ]

veh_types-own [ an/truck personal ]

turtles-own []

; property breeds
breed [ nodeIDs nodeID ]
breed [ edgeIDs edgeID ]

; property variables
nodeIDs-own [ long lat prop_type prop_grp_ID propertyID ]

; global variables
globals [ current-time total-time ]

I’ve also tried playing with this code to try and create the nodes separately, but to no avail:

  ; build a list or agentset containing the nodes
  let node-list []  ; creates an empty local variable list
  nw:load-graphml "practice_graph.graphml" [
    set node-list lput self node-list  ; creates a global node-list that adds the value (self) into the node-list variable
  ]
  let node-set turtle-set node-list

The newest code i’ve tried as of last night is:

extensions [nw gis]

directed-link-breed [ personals personal ]
directed-link-breed [ animals animal ]

;personals-own [ weight time vehID ]
;animals-own [ weight time vehID ]

links-own [ d8 d5 d6 ]

turtles-own [ propertyID prop_grp_ID prop_type long lat ]  

globals [ total-time current-time ]

;;;;;;;;;;;;;;;;;;;;
;;;;SETUP;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;

to setup
  clear-all
  
  gis:set-transformation [-180 180 -90 90 ] [-10 10 -5 5 ]
  
  nw:set-context turtles links
  nw:load-graphml "practice_graph.graphml" 
  
  ask turtles [ setxy long lat ]
  
  ;create-turtles-from-links
  
  set total-time 19843
  set current-time 19037
  
  visualize
  
  reset-ticks

end

Any direction or help would be greatly appreciated. thank you!

[ERSUCC]

Hi @pandora,

Correct me if I misunderstood, but it seems like the main problem you’re having is that edges in the GraphML file are imported into NetLogo as links instead of turtles. I don’t think there’s a way to directly import edges as turtles, however you can convert the links into turtles after importing the graph. Here’s one way you could approach this:

breed [ vehicles vehicle ]

to setup
  ...

  nw:load-graphml "practice_graph.graphml"

  ask links [
    ask end1 [
      hatch-vehicles 1
    ]

    die
  ]

  ...
end

Hopefully this helps! Let me know if that doesn’t solve the problem, or if you have any other questions.

Isaac B.

[pandora]

Hi Isaac,

Thank you. Your help is very much appreciated.

I believe you have it correct. each edge in my network is a movement of the vehicle between nodes. I want to track those vehicles as an epi model for disease tracking. therefore, those vehicles, can only move on the edges that are designated for them. I’ve updated the model based on another feedback about what breeds really are from a different post. And added your code. I think what your code is saying is- ask each link and find out it’s source , then create one vehicle at that node. the kill that link. Is this correct? I actually don’t think I want to kill the link- bc i need to know the target node and have that vehicle move to that target next. so it needs to follow it’s edges. Does that make sense?
Also, when I load this, the nodes are not placed at the lat and long- just all bunched up. Do you have any advice about this.

[ERSUCC]

Hi @pandora,

You can certainly leave the links in place, but if you would prefer, you could also add vehicles-own variables to keep track of end1 and end2 of the original link after it dies.

Regarding the node positions: it looks like you’re trying to set a transformation with the gis extension, but this only affects data imported directly with the gis extension, it’s not a global transformation for world coordinates. What I would recommend instead is to calculate the bounds of the imported data (eg. with min [ lat ] of nodes, etc.) and then manually scale the latitudes and longitudes before calling setxy. Or if you already know what the bounds will be, then you don’t need to do anything with min and max, and you can just scale the positions directly.

Let me know if any of this doesn’t make sense, or if you need help implementing that behavior!

Isaac B.

[pandora]

Hello @ERSUCC

That gave me an idea. Instead of worrying about edges at all, why not just import the vehicles as turtles and use the source, target nodes and just have them move. Does it have to follow an edge??

I also tried importing everything from a csv (vehicle list, edge list and node list). I got something this way, though much longer to load with over 430,000 ‘edges’ and 1900 vehicles.

FYI- the gis worked. I was able to import and graph the nodes by looping through the rows of the node list and using the gis coordinates with this code. I’m just having trouble with the vehicles now, when i try to assign the vehicle x/y coord to the first node it starts on. I might start a new post unless you don’t mind me following up with that here?

thank you for helping me through this, your insights really helped.

[ERSUCC]

Hi @pandora,

It’s good to hear that you’re making progress! We can continue in this issue or in a separate issue, it doesn’t really make a difference as long as it’s related.

I’m not sure what you mean by “just import the vehicles as turtles”. If you’re able to edit your GraphML file to represent the vehicles as nodes, then it definitely would make more sense to create them as turtles from the start. But if the vehicles will always be represented as edges in the GraphML file, I don’t think you will be able to import them as turtles, because the nw extension assumes that nodes correspond to turtles and edges correspond to links.

Could you be a bit more specific about the issue you’re having with gis? When you try to assign the x/y coordinate, what exactly happens that is unexpected?

Isaac B.

[pandora]

I appreciate it. Here is my output so far with code. I went back to the data and made 3 csv files, one for nodes, one for edges, and one for vehicles. And I uploaded each csv so that the nodes and vehicles were turtles. I had the edges there too, but not sure if I really need them or not since the vehicles will be moving between the nodes anyway with the same information that is in the edge list. The problem right now, is the code part that says ;fin the corresponding source node and place vehicle there. I’m getting the runtime error : NODES breed does not own variable START.

extensions [nw csv gis]

breed [ nodes node ]
breed [ cars car]

nodes-own [ nodeID propertyID prop-grp-id prop-type long lat ]
cars-own [ vehID vtime start finish vtype ]

globals [ current-time total-time ]

; These are the variables in each csv file
;nodeID,propertyID,prop_grp_ID,prop_type,long,lat
;linkID,time,source,target,vehID,weight
;vehID,time,from,to,veh_type

;;;;;;;;;;;;;;;;;;;;
;;;;SETUP;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;

to setup
  clear-all
  
  gis:set-transformation [-180 180 -90 90 ] [-10 10 -5 5 ]
  
  set current-time 18927
  set total-time 19843
  
  ;load-nodes
  load-nodes "node_list2.csv"
  
  ; load-vehicles
  load-cars "vehicle.csv"
  
  reset-ticks

end

;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;MAIN PROCEDURE;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;

to go
  if current-time >= total-time [ stop ]
    set current-time current-time + 1  
  tick
end 


;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;PROCEDURES;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;

to load-nodes [node_list2.csv]
  file-open "node_list2.csv"
  while [not file-at-end?] [
    let values csv:from-row file-read-line  ;convert line to a list of values
    create-nodes 1 [
      set nodeID item 0 values ; assumes the first column is an ID
      set propertyID item 1 values
      set prop-grp-id item 2 values
      set prop-type item 3 values
      set long  item 4 values
      set lat  item 5 values
      
      setxy long lat
      set shape "house"
      set size 0.2
      set color orange
      ;print (word "Node created with ID: " nodeID long lat)
    ]
  ]
  file-close

end


to load-cars [vehicle.csv]
  file-open "vehicle.csv"
  while [not file-at-end?] [
    let values csv:from-row file-read-line
   
    ;let vehicle-time item 1 values ; have to read the time in before the if statement
    ;if vehicle-time = current-time [
      create-cars 1 [
        set vehID item 0 values
        set vtime item 1 values
        set start item 2 values
        set finish item 3 values
        set vtype item 4 values
      
        set shape "car"
        set size 0.3
        set color white
        
        print (word "Loading Car: " vehID " with startID: " start " at time: " vtime " Current " current-time )
        
        ; Find the corresponding source node and place vehicle there
        let source-node one-of nodes with [nodeID = start]
        if source-node != nobody [
          setxy [xcor] of source-node [ycor] of source-node  ; Place vehicle at the source node
        ]
      ]
    ]
  ;]
  file-close
end
```

I’m trying to start the vehicles on the starting node that it starts on from the csv file.
Ex:

vehicle name Start_node end_node time (of that movement)

So I’m trying to find the node that equals the start_node number to place the vehicle there at that time moment. - I haven’t gotten the ticks to work yet-

then each tick, the model should check the next time in the data and create that vehicle movement between the nodes.
this will eventually be an epi model where the vehicles will be the transmission between the nodes, etc.

[pandora]

Oh, and I have tried to use ‘read-from-string’ command, but made sure that the data is numeric, etc.

[ERSUCC]

Hi @pandora,

The with primitive doesn’t look in the parent agent context to find variables, so you’ll have to store the value of start in a local variable for it to be found. For example:

create-cars 1 [
  ...

  let local-start start
  let source-node one-of nodes with [ nodeID = local-start ]

  ...
]

You can also use myself to explicitly reference the parent context, for example:

create-cars 1 [
  ...

  let source-node one-of nodes with [ nodeID = [ start ] of myself ]

  ...
]

Isaac B.

[pandora]

Now that makes so much more sense. I was wondering why there were so many local variables made in the code snippets I was looking at. And that just worked perfectly. Now i’ll get the ticks working and hopefully move on to movement. I don’t even know where to start to thank you. I very much appreciate your help and explanation.

[ERSUCC]

Good luck, and feel free to open a new issue if you have any more problems!