Design Target Achievement Index

A Differentiable Metric to Enhance Deep Generative Models in Multi-Objective Inverse Design

Lyle Regenwetter1, Faez Ahmed1

1MIT 

Explore

DTAI is a project centered on incorporating target achievement and feasibility into the training of Deep Generative Models. Get started exploring FRAMED's data, and the research directions that FRAMED enables:

Defining the DTAI Score

The DTAI score is a differentiable, bounded, and easy to evaluate target satisfaction index that can serve as a training objective (loss) or an evaluation metric for generated designs. Click on the images below to learn about the score's design and the impact of DTAI's tuning parameters.

The DTAI score rewards designs that exceed performance targets and penalizes designs that underperform performance targets. To evaluate the target achievement performance of a design, we calculate an individual score for each performance objective based on the performance to target ratio in that objective. Excessive overperformance nets decreasing rewards, while excessive underperformance causes constant score penalties.

As designers, we may often have certain objectives that are significantly more important than others. Alpha adjusts the importance of a particular performance objective relative to others by adjusting the steepness of the DTAI curve.

Once a design's performance has exceeded the performance target, further improvement becomes less valuable. However, in certain objectives, the value of further improvement may be greater than in others. For example, we may like to continue to decrease the weight of our bicycle beyond our design target, but further improving the safety factor beyond the target has less value. Beta adjusts the steepness of the score falloff, allowing designers to account for this nuance.

Training Performance Target, Feasibility, and Diversity-Aware Deep Generative Models

We proposed a method to simultaneously incorporate feasibility, diversity, and performance target achievement into the training of Deep Generative Models. First, performance and feasibility predictors are trained using datasets of feasible designs, infeasible designs, and performance values for the feasible designs. Next, given a batch of samples generated by a Deep Generative Model, such as a GAN, performance estimates and feasibility likelihoods are calculated using the performance estimators. Performance values are aggregated using the DTAI score and scaled by feasibility likelihood for a set of feasibility-weighted performance scores. Finally, we borrow the diversity scoring from PaDGAN using a diversity kernel and Determinantal Point Process (DPP). This final performance, feasibility, and diversity loss are appended to the GAN's generator loss function.

As long as the predictive models used are differentiable, our final loss is differentiable since the DTAI score, Similarity Kernel, and DPP is differentiable. This makes our proposed framework compatible with Deep Learning-based generative models, which rely on gradient decent, requiring that training losses be fully differentiable.

Benchmarking Performance

We evaluate the proposed framework's performance across eight metrics against several benchmark methods. The proposed feasibility, novelty, and DTAI-augmented GAN outperform baselines across the board, achieving the highest performance in all metrics except diversity. All metrics are defined in our paper.

Distribution GAP

Not only are average scores across generated designs superior in the proposed framework, distributions across designs drastically outperform baselines.

Ablation Study

The proposed DTAI score and auxiliary feasibility classifier hold up in ablation studies. When tested without the DTAI score, without the classifier, or without both (The case without either is the standard MO-PaDGAN), the proposed framework generally outperforms or competes in every objective.

Citations

Chicago

Regenwetter, Lyle and Faez Ahmed. "Design Target Achievement Index: A Differentiable Metric to Enhance Deep Generative Models in Multi-Objective Inverse Design.” In Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, {IDETC-22}, St. Louis, MO, 2022.

Bibtex

@inproceeedings{regenwetter2022design,
    title={Design Target Achievement Index: A Differentiable Metric to Enhance Deep Generative Models in Multi-Objective Inverse Design},
    author={Lyle Regenwetter and Faez Ahmed},
    year={2022},
    booktitle={International Design Engineering Technical Conferences and Computers and Information in Engineering Conference,
    {IDETC-22}},
     organization={ASME},
     day = {14-17},
     month = {Aug},
     address = {St. Louis,
     MO},
     year={2022}}

ACKNOWLEDGEMENT

We would like to thank Amin Heyrani Nobari for creating the Tensorflow 2.x version of PaDGAN, which we modified to generate our results. We also acknowledge MathWorks for supporting this research.