Deep graph generative models have recently received a surge of,attention due to its superiority of modeling realistic graphs in a,variety of domains, including biology, chemistry, and social science.,Despite the initial success, most, if not all, of the existing works are,designed for static networks. Nonetheless, many realistic networks,are intrinsically dynamic and presented as a collection of system,logs (i.e., timestamped interactions/edges between entities), which,pose a new research direction for us: how can we synthesize realistic,dynamic networks by directly learning from the system logs? In,addition, how can we ensure the generated graphs preserve both,the structural and temporal characteristics of the real data?,To address these challenges, we propose an end-to-end deep,generative framework named,TagGen,. In particular, we start with a,novel sampling strategy for jointly extracting structural and temporal context information from temporal networks. On top of that,,TagGen,parameterizes a bi-level self-attention mechanism together,with a family of local operations to generate temporal random,walks. At last, a discriminator gradually selects generated temporal,random walks, that are plausible in the input data, and feeds them,to an assembling module for generating temporal networks. The,experimental results in seven real-world data sets across a variety of,metrics demonstrate that (1),TagGen,outperforms all baselines in the,temporal interaction network generation problem, and (2),TagGen,significantly boosts the performance of the prediction models in,the tasks of anomaly detection and link prediction.