CODESIGN.BLOG

DESIGN THE WORLD

Menu
  • HOME
  • LIVE STREAMING
  • PROJECTS
  • COMMUNITY
  • EVENTS
  • TEAMWORK
  • RESEARCHERS
  • TOOLS
  • FAQ
  • ABOUT
  • CONTACT

DESIGN DEV-OPS SERVICE

AI Week TAU 2021

VE’AHAVTA PROJECT

https://codesign.blog/wp-content/uploads/2018/07/veahavta-project.mp4

VE’AHAVTA PROJECT

OUR FB GROUP

‏WHOLE WORLD OF DESIGN‏
‎‎Public‎ group‎
JOIN TO OUR GROUP
The whole world of design is a roof group that concentrates all the information on graphic design, industrial design, visual communication, interior design, fashion design, architecture, environmental design and more.

Here you will find trends in design, technological innovation, job offers, professional recommendations, training, technological aids, start-ups companies and more.
 

OUR FB GROUP

‏STARTUPS IN ISRAEL‏
‎‎Public‎ group‎
JOIN TO OUR GROUP
The group's goal is to disseminate and share vital science in all aspects of their technological ecosystem in Israel, and to help create connections between entrepreneurs, private investors and venture capital funds.

For any questions, cooperation and assistance for members of the technological community, please contact us with comments or details. Good luck everyone!
 
AI Data Science Summit 2020

COVID-19 AND AI: A VIRTUAL CONFERENCE

COVID-19 and AI: A Virtual Conference

DATABASE FOR RESEARCHERS TEAMS: FIGHT THE CORONA EPIDEMIC

Database for Researchers and Thinking Teams

NOVEL CORONAVIRUS INFORMATION CENTER

NOVEL CORONAVIRUS INFORMATION CENTER

NOVEL CORONAVIRUS COVID-19 (2019-NCOV) DATA REPOSITORY

Novel Coronavirus COVID19 Data Repository

INTERACTIVE MAPS OF CORONAVIRUS CASES IN ISRAEL

Interactive Maps of Coronavirus Cases in Israel

ON EXIT STRATEGY FROM COVID-19 LOCKDOWN

Live Webinar Exit Strategy COVID19

Cyber Week TLV 2021

Cyber Week TLV 2021

The NANO.IL.2021 Conference & Exhibition JLM

The NANO.IL.2021 Conference & Exhibition JLM

ACCV 2020

ACCV 2020

IMVC 2020

Israel Machine Vision Conference (IMVC) 2020

AIIDE 2020

AIIDE 2020

ICDES 2020

ICDES 2020

Service Design Hong Kong 2020

Service Design Hong Kong 2020
Home » Data Science » Design » Engineering » Explaining of Israel » Technology » tNodeEmbed — Node Embedding over Temporal Graphs

tNodeEmbed — Node Embedding over Temporal Graphs

August 19, 2019 sherry 3 Data Science, Design, Engineering, Explaining of Israel, Technology,

By Uriel Singer


In this post I’ll take about: tNodeEmbed (joint work with Dr. Ido Guy & Prof. Kira Radinsky). tNodeEmbed is a method that leverages the temporal information in graphs in-order to create richer node embedding representations. This concept was written up as a paper which was accepted to IJCAI2019.
Let’s start talking about embeddings in the graph domain.

Embeddings in the graph domain

They are many ways to do embeddings in the graph domain:

Figure 1: Different types of embeddings in the graph domain

In Figure 1 we can see that it is possible to embed the whole graph, embed each node and embed each edge (one naive way is by merging it’s two node embeddings).
All of these embeddings can be then used in-order to answer different ML questions.

Temporal graphs hold rich information that can’t be captured from a static graph. For example, suppose Eve became Facebook friends with Alice and Bob in the same day, while she became friends with Chris a year ago. Who is more likely to become friends with Alice? is it Bob or Chris? In a static graph there is no difference (4 nodes with 3 edges: Eve to Alice, Bob and Chris). A temporal graph holds the information on when each edge occurred, which resolves us with the conclusion that Alice is more likely to become friends with Bob than Chris.

The Idea behind this paper is learning Node Embedding over Temporal Graphs:

Figure 2: creating node embeddings using the temporal information of the graph.

Figure 2 illustrates the idea behind node embedding over temporal graphs. We define the graph in time t to be the graph that holds all edges which appeared at any point up to time t.

tNodeEmbed

Lets start by overviewing tNodeEmbed:

Figure 3: Overview of tNodeEmbed

tNodeEmbed consists from 3 steps:

1. Initialization

Several prior works studied node embedding and reported good performance on static graphs. The architecture presented in this paper can use any of the known approaches for node embedding over static graphs. Specifically, we opted to work with node2vec, which achieved state-of-the-art performance on various benchmarks.

Figure 4: node2vec overview

In Figure 3 we can see the node2vec overview, which is simply a reduction from the graph domain to the text domain: Each node is treated as word while the ‘sentences’ are generated by sampling many random walks. Those ‘sentences’ are inserted into a word2vec model. The word embeddings outputs are returned to the graph domain as the nodes embeddings.

We use node2vec to initialize Qt. Specifically, we compute the node embeddings for all T graphs Gt1, …, GtT. The outcome of this stage is a d X Tvector per node, where T is the number of time steps and d is the embedding size (each column is a different static embedding).

2. Temporal Node Embedding Alignment

In-order for us to be able to look on how a specific node moved between the time-steps in the embedding space, we still need to align the time-steps embeddings.

Figure 5: Illustration of the alignment between 2 consecutive time-steps

In order to do so, we apply Orthogonal Procrustes on every two consecutive time-steps iteratively. The assumption we use here is that the 2 embedding spaces are close to each other enough for them to be aligned by a rotation matrix. This is a reasonable assumption due to that we do it on 2 consecutive time-steps that differ only in the edges that were addedbetween the two time-steps.

For people that find this technique interesting, here’s another research by FAIR which uses the same approach for unsupervised NMT.

3. Node Embedding over Time

In the final step we merge the d X T vector of each node to a d vector using a LSTM. The LSTM is then connected into an additional layer that tries to predict a specific task is can be seen in Figure 6.

Figure 6: The different final architectures of tNodeEmbed

Results

We compared tNodeEmbed to various baselines, both static and temporal, and on a various different datasets. Two tasks we experimented on are the temporal link-prediction and multi-label node classification tasks:

1. Temporal Link Prediction

In-order to create this task, we picked a pivot-time step in such a way that all edges that are known up to that pivot time-step are used for training while the edges that are known after that pivot time-step are used for test.

Table 1: Link prediction results

An interesting observation is that the lower the clustering coefficient (CC), the higher the gap between tNodeEmbed to its static baseline, node2vec.
We understand from this phenomena that the “denser” the graph, the less informative the temporal information is in-order to predict future links. The reason lies in the fact that edges tend to attach to higher degree nodes, and in “denser” graphs, these nodes have extremely high degrees compared to other nodes.

2. Multi-Label Node Classification

Table 2: Multi-label node classification results

In Figure 2 we can observe the tNodeEmbed does pretty well.

Final Words

tNodeEmbed nailed it!

Thank you for reading. Feel free to use this work or continue the research and contribute to the graph embedding community!


Important Links

The paper: https://www.ijcai.org/proceedings/2019/0640.pdf
The code: https://github.com/urielsinger/tNodeEmbed

0 0 votes
Article Rating

Tags: A temporal graph, academia, administrative perspectives, AI, AI methods, analytics, analytics space, applied machine learning, approach, architecture, artificial intelligence, assumption, Automation of Machine-Learning, Autonomous driving, autonomous ship, Big Data Analytics, broad spectrum, classification tasks, clustering coefficient (CC), code, comments, complex maneuver, concept, conclusion, Conference, connected, consecutive time-steps, content, Continuous delivery, contribute to the graph embedding community, correct angle, cost-effective opportunity, Cyber, Data Science, Data Scientists, deep learning, delivering, DESIGN, Developers, DevOps, Dialogue Bots, different ML questions, Different types, directly, domains, Dr. Ido Guy, Earth, ecological aspect, Education, embeddings, engineering, equal, Ethics of artificial intelligence, excellent, experience, Explaining of Israel, Facebook, FAIR, Final Words, Fintech, focus, friends, Future of AI, graph domain, great success, Healthcare, helpful feedback, high-quality, higher degree nodes, idea, ideas, IJCAI2019, Illustration of the alignment between 2 consecutive time-steps, Important Links, improvement, industry, industry tracks, Initialization, innovation, Intelligent robots, iot, iteratively, land, Large scale analytics, launched at the beginning, leading experts, Learning, Link prediction results, LSTM, Machine ethics, moon’s gravity, Multi-Label Node Classification, Multi-label node classification results, NASA, National Aeronautics and Space Administration, Natural Language Understanding, Node Embedding, NODE EMBEDDING OVER TEMPORAL GRAPHS, Node Embedding over Time, node2vec, node2vec overview, observe, organizing conference, Orthogonal Procrustes, other nodes, Overview of tNodeEmbed, overviewing, paper, participants, performance on static graphs, performance on various benchmarks, phenomena, pivot, Pray for the success of Genesis!, predict future links, predictive applications, Prof. Kira Radinsky, reading, real world, real-world domains, Registration, reinforcement learning, Research, research and application, research innovations, research track, researchers, results, Retail, richer node embedding representations, Robot rights, rotation matrix, space industry, spacecraft, SpaceX, sponsorship, state-of-the-art, static graph, studied, Systems for ML, technical presentations, technique interesting, TECHNOLOGY, Temporal graphs, temporal information, temporal information of the graph, Temporal Link Prediction, Temporal Node Embedding Alignment, test, text domain, The conference, The different final architectures of tNodeEmbed, The People of Israel, The Summit, the tNodeEmbed, Threat to human dignity, time-step, time-steps embeddings, TNODEEMBED, tNodeEmbed nailed it!, topics, tutorial, two consecutive time-steps, unsupervised NMT, Uriel Singer, various baselines, Weaponization of AI, week of lunar orbits, word2vec model

Previous: All presentations from the UAI 2019 event
Next: The Evening of the MDLI Course — 1+2 Cycles

Related posts

  • Fairness Accountability Transparency and Ethics in Computer Vision Tutorial

    CVPR 2020 • FATE in Computer Vision

    Tutorial on Fairness Accountability Transparency and Ethics in Computer VisionOverviewComputer vision has ceased to be a purely academic endeavor. From law enforcement, to border control,...
  • CVPR 2020 Learning 3D Generative Models Workshop

    CVPR 2020 • Learning 3D Generative Models Workshop

    Deep Declarative NetworksIntroductionThe past several years have seen an explosion of interest in generative modeling: unsupervised models which learn to synthesize new elements from the...

About The Author

sherry
sherry

CODESIGN.BLOG

Subscribe
Notify of
guest
guest
3 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
ETIAS
ETIAS
3 years ago

Why visitors still use to read news papers when in this technological world the whole
thing is available on web?

0
Reply
seogiare.vn
seogiare.vn
3 years ago

I could not resist commenting. Well written!

0
Reply
seogiare.vn
seogiare.vn
3 years ago

If you are going for finest contents like myself,
simply pay a quick visit this site every day because it presents feature contents, thanks

0
Reply
wpdiscuz   wpDiscuz

ARTIFICIAL INTELLIGENCE REVOLUTION 2020

Artificial Intelligence Revolution 2020

WHY WOULD THE FIELD OF DATA SCIENCE BE THE MOST SOUGHT AFTER IN 2020?

WHY WOULD THE FIELD OF DATA SCIENCE BE THE MOST SOUGHT AFTER IN 2020?

Cyber Week TLV 2021

Cyber Week TLV 2021 Rescheduled

CVPR 2020 • LEARNING 3D GENERATIVE MODELS WORKSHOP

Learning 3D Generative Models Workshop

CVPR 2020 • FATE IN COMPUTER VISION

Fairness Accountability Transparency and Ethics in Computer Vision Tutorial

E&PDE HERNING 2020

E&PDE HERNING 2020

European Conference on Computer Vision 2020

European Conference on Computer Vision 2020

The NANO.IL.2021 Conference & Exhibition JLM

The NANO.IL.2021 Conference & Exhibition JLM

ICED 21 • DESIGN IN MOTION

ICED 21 • DESIGN IN MOTION
AI Data Science Summit 2020

Recent Comments

  • AlisonFargo on CVPR 2020 • FATE in Computer Vision
  • Ahmderomo on DeepMind’s AI can ‘imagine’ a world based on a single picture
  • Ahmderomo on The 35th Conference on UAI
  • Ahmderomo on Machine Learning Roadmap
  • Online Lenders on The Boy Robot of 1774
  • Quick Loans on Moda Ani – Hillel Band
  • Danielden on The Boy Robot of 1774
  • เลขอั้น on Design for a Vision
  • carmellapm18 on CVPR 2020 • Learning 3D Generative Models Workshop
  • filmkovasi on CVPR 2020 • FATE in Computer Vision

Archives

  • April 2021
  • December 2020
  • October 2020
  • June 2020
  • April 2020
  • March 2020
  • January 2020
  • December 2019
  • November 2019
  • October 2019
  • September 2019
  • August 2019
  • July 2019
  • June 2019
  • May 2019
  • April 2019
  • March 2019
  • February 2019
  • January 2019
  • December 2018
  • November 2018
  • October 2018
  • September 2018
  • August 2018
  • July 2018
  • June 2018
  • May 2018
  • April 2018
  • March 2018
  • February 2018
  • January 2018
  • December 2017
  • November 2017
  • October 2017
  • September 2017
  • August 2017
  • July 2017
  • June 2017
  • May 2017

TAGS

academia AI analytics analytics space applied machine learning artificial intelligence Conference Cyber Data Science Data Scientists deep learning DESIGN Developers DevOps domains Education engineering Ethics of artificial intelligence experience Explaining of Israel Fintech focus great success Healthcare ideas industry industry tracks innovation iot leading experts MACHINE LEARNING predictive applications real world Registration reinforcement learning researchers research innovations research track Retail state-of-the-art technical presentations TECHNOLOGY The conference The Summit tutorial

Copyright © 2023 | designed by share wood studios

wpDiscuz