Review of cloth modeling and simulation for virtual fitting

Abstract

Virtual fitting technology is widely used in many fields, such as e-commerce, garment computer-aided design, and film and game production. Cloth modeling and simulation play an important role in virtual fitting. The realism and simulation speed directly affect the user experience and visual appearance. This paper first reviews the history of cloth modeling and simulation methods. Then, it focuses on four perspectives: yarn-structure modeling of the cloth modeling level; multi-resolution grid simulation; the relation between human posture and cloth deformation; and collision problems of the cloth simulation level. Yarn-structure modeling considers a unit cell that incorporates the physical characteristics of the fabric and the fabric structure. A multi-resolution grid can integrate techniques from different research fields and may achieve a breakthrough in cloth modeling. Generative adversarial networks (GANs) have potential for dealing with the relation between human posture and cloth deformation by comparing several typical algorithms with GANs. Solving the collision and friction problem depends on choosing the right envelope box for refining the human body surface. Finally, the paper concludes with an analysis of future research trends that could improve the fidelity and speed of a simulation. The paper may serve as a reference for research into cloth modeling and simulation for virtual fitting.

Keywords

Virtual fitting cloth simulation yarn structure multi-resolution grid human posture cloth deformation collision and friction

Virtual fitting is an important aspect of garment computer-aided design (CAD), as it allows a virtual model of a consumer to “wear” virtual clothes that fit their body shape. The consumer can observe the worn clothes from different angles. This technology can be used in various fields, such as e-commerce, film and game production, and smart manufacturing, which would allow better personalization by the apparel industry.^1–3 Virtual fitting mainly involves three-dimensional (3D) body modeling technology, cloth modeling and simulation technology, real-time dressing technology, human posture tracking technology, etc. This paper focuses on the analysis of the key technology of cloth modeling and simulation technology, which currently has the problems of insufficient cloth texture display, a complex cloth modeling process, a large number of simulation calculations, insufficient simulation effect of cloth deformation with the human posture, and the cloth penetration and super elasticity phenomenon, which need to be improved.⁴

Researchers have conducted research into the real-time visual simulation of clothes to allow virtual fitting.^5,6 There are several comparative reviews of these studies. Liang et al.⁷ presented a number of perspectives on cloth properties, cloth modeling methods and history, numerical simulation, human modeling, collision response, garment design software, and touch perception technology. A review by Min et al.⁸ for cloth animation described seven categories of technical approaches. Qinting et al.⁹ reviewed research on physical modeling and simulations of cloth based on mass–spring models. They focused on the methods, advantages, and disadvantages of model optimization and system stability, and the mitigation for over-elasticity. Mozafary and Payvandy¹⁰ analyzed and compared existing methods for dealing with over-elasticity and collision detection, and its response for simulations with a mass–spring model. On the other hand, Meng et al.¹¹ and Aijing and Jie¹² focused on the key technologies of 3D body scanning and modeling. They created garment libraries, and simulated cloth for virtual fitting systems. These authors compared the characteristics and visualization of several virtual fitting systems, and briefly described the principles, advantages, and disadvantages of geometric modeling, physical modeling, and hybrid modeling at the cloth simulation level. Hu P et al.¹³ presented a virtual fitting approach for wearable clothing and proposed their own generic approach to virtual fitting. Yanxia et al.¹⁴ described the history of cloth modeling, including geometric modeling, physical modeling, and hybrid modeling. They reviewed optimized approaches such as the numerical integration method and multi-resolution grid method, which is more systematic and more detailed. Ge and Xiaohui¹⁵ provided an overview of design methods for digital garment modeling, including the introduction of two-dimensional (2D) paper patterns, 3D virtual design, virtual sewing, and other techniques. As we can see from the above introduction, Liang et al.⁷ presented many perspectives, resulting in a lack of depth and breadth in each perspective. Min et al.⁸ confined themselves to an overview of cloth animation. Both Qinting et al.⁹ and Mozafary and Payvandy¹⁰ reviewed only one method for building a mass–spring model of cloth, which was relatively incomplete. Meng et al.¹¹ and Aijing and Jie¹² did not provide a comprehensive or in-depth analysis of the three modeling methods reviewed. There were limitations in the methodological presentation of virtual try-ons by Hu P et al.¹³ Yanxia et al.¹⁴ neglected to summarize the collision and friction problem, which is an important aspect of fabric simulation. Ge and Xiaohui¹⁵ focused only on the presentation of the current state of the art in apparel modeling design for the apparel designer application population. Moreover, none of these articles specifically summarized or analyzed the increasingly popular research on yarn-structure modeling and the relation between human posture and garment deformation. Moreover, their discussions of the current progress in the research progress were not comprehensive enough.

Through extensive literature reading and analytical research, we have identified some popular research perspectives in recent years and their help in addressing the current inadequacies of cloth modeling and simulation technology. Firstly, researchers have investigated yarn modeling methods that reflect cloth texture effects, such as Li et al.,¹⁶ Xu HY et al.,¹⁷ Sha and Gaoming,¹⁸ and Xin et al.,¹⁹ with particular contributions to the modeling of knitted fabric yarns, which is useful for the problem of insufficient fabric texture effects. Secondly, researchers have investigated the theory of a multi-resolution grid in cloth simulation processes, for example, Narain et al.²⁰ and Tamstorf et al.,²¹ and the method has provided ideas for simplifying the computational volume of the simulation. Thirdly, researchers have also launched studies on virtual fitting based on human posture, such as Han X et al.⁵ and Bowen et al.,²² which helped to optimize the effect of simulation following posture for real-time fitting. Finally, the collision problem has been studied by researchers early on, such as Ling et al.,²³ Yong et al.,⁶ and Chen et al.,²⁴ and a variety of theoretical approaches have been proposed to reduce the computational effort and solve the cloth penetration problem. The cloth modeling and simulation technology is made up of multiple links, of which these four perspectives have research coherence and are processed sequentially. It is due to the hotspots of interest for researchers, the inadequacy of optimized cloth modeling simulation techniques, and the coherence of the research links that this paper will review these four perspectives, namely yarn-structure modeling, multi-resolution grid simulation, the relation between human posture and cloth deformation, and collisions and friction.

Therefore, this paper focuses on cloth modeling and simulation for virtual fitting as a review. To provide a more comprehensive and deeper analysis of the progress of research, this paper summarizes the methods that have been widely researched by scholars in recent years.

In the introduction, we analyze relevant review articles, and the ideas of analysis and discussion from the four perspectives are elaborated.

In the Cloth modeling and simulation section, this paper focuses on four perspectives: yarn-structure modeling of the cloth modeling level, multi-resolution grid simulation, the relation between human posture and cloth deformation, and collision problems of the cloth simulation level. We summarize and analyze the research ideas, theoretical parameters, simulation effects, computational quantities, etc., and also discuss and compare the advantages and limitations of the different methods.

In the Analysis and discussion section and Conclusions section, this paper presents insights on the research trends based on the four perspectives of yarn-structure modeling, multi-resolution grid simulation, the relation between human posture and cloth deformation, and collisions and friction. We also provide an outlook on future developments in cloth modeling and simulation for virtual fitting.

Cloth modeling and simulation

Cloth is a typical flexible material. It is highly moldable and undergoes a wide range of deformations.²⁵ Problems with modeling and simulating various mechanical properties can directly affect the realism of the simulated cloth in real-time virtual fitting. After years of in-depth research, a variety of cloth modeling and simulation methods have been developed and various outstanding problems have been identified. This section will focus on four aspects in turn for an in-depth analysis: yarn-structure modeling, multi-resolution grid simulation, the relation between human posture and cloth deformation, and the collision and friction problem.

Yarn-structure modeling

Researchers have successively used geometric modeling, physical modeling, and hybrid modeling to simulate cloth.²⁶ Geometric modeling uses a mesh and fits the drape curve. It is simple and convenient, but because the physical and mechanical properties of the cloth are ignored, the simulation is not realistic.

Therefore, since the late 1980s, researchers began to study physical modeling methods, among which the mass–spring model established by Provot²⁷ has been widely used. This models the cloth as a collection of masses, and the mutual mechanical relations between the masses are represented as flexible springs, structural springs, or shear springs. Although the results for this method are relatively realistic, the computational process is very time-consuming.

Next, researchers started to use a hybrid modeling approach, which is a blend of geometric and physical modeling.²⁸ The computation time is shorter due to the lower accuracy, but the overall simulation duration and visualization are still poor so the method is not widely used. Because of these problems with the traditional modeling methods and the difficulties in representing the characteristics of the yarn twist, fabric structure, and fiber texture, researchers have begun to adopt yarn-structure modeling, which can realize texture realistically when simulating cloth.²⁹ This method takes yarn as the basic unit and uses the interaction between threads to simulate the deformation of the cloth. It can represent the structural characteristics of the fabric, and the simulated fabric appears more realistic.³⁰

Three-dimensional modeling of a single yarn

In 2005, Chu used B spline curves to simulate yarn,³¹ while later Kaldor et al.³² defined yarn motion and embodied the yarn mechanical properties to obtain a dynamic simulation of cloth based on the fabric structure. However, this method could output stable results only by using a small time step. For 3D single-yarn modeling, Zheng³³ in 2010 used a uniform secondary B spline curve to design the yarn cross-section and designed the yarn centerline from key control points. The yarn path contour was fitted using a shape-preserving uniform tertiary B spline curve. Finally, random turbulence was added to achieve a realistic 3D twist effect for a single yarn. Figure 1 compares the results of the simulation with a photograph of the yarn. However, the modeling process is complicated and it is not easy to extract information about the formed fabric surface. Similarly, in 2020 Peng et al.³⁴ used the quadratic Bézier curve described by Zheng³³ to construct the yarn geometry. One difference was that the 3D yarn surface texture mapping was achieved by calculating yarn surface points and the corresponding texture coordinates to map the 3D yarn, which was processed by surface deformation to realize the yarn simulation. Other 3D single-yarn models have been built by sweeping through the path to form the fabric plane when the equation for the midline of the yarn is known, such as the aforementioned methods of Chu et al.³¹ and Zheng,³³ and also the method proposed by Zhang et al. in 2017.³⁵

Figure 1.

Comparison of simulation results (a) with a photograph of actual yarn (b).³³

Three-dimensional modeling of a yarn unit structure

By considering yarn unit structures and their physical properties, current research has led to lightweight methods with improved simulation efficiency and enhanced realism. In 2020, Ji et al.³⁶ incorporated tube geometry and three.js into a spline curve to obtain a novel 3D yarn unit model that was lightweight and quick. It can be used for simulating the deformation of a complex structure, such as a 3D warp knitted fabric, but does not consider twisting or mechanical properties.

In terms of research based on physical properties, in 2016, Dong-Yong et al.³⁷ used a fiber block as a unit to simulate yarn and cloth. They developed a mass–spring structure with key fiber points and used twisting kinematics and kinetic equations to simulate the twisting of two single threads to build a fabric model with fiber texture. The resulting simulation was more detailed and more realistic, but the number of particles and, thus, the simulation time were higher than for the traditional method. However, the above two methods both ignore the deformation of the yarn cross-section due to inter-fiber interactions, as they used an idealized fixed shape for the yarn cross-section.

In 2020, Ying et al.³⁸ proposed a cloth modeling method based on digital cell theory in which the yarn fibers were discretized. They derived a method for calculating the inter-fiber friction. The cloth was modeled accurately, but with an increase of the resolution of the discretization of the yarn fibers, the simulation time increased significantly.

In 2020, Grechukhin et al.³⁹ used nonlinear bending theory and proposed a method based on a preliminary calculation of the fabric structural parameters to realize a 3D model of clothes. Figure 2 shows a mesh bandage simulated using this method when worn on a human head. However, the mathematical model in this method is applicable only to fabrics woven from yarns with a low compression rate.

Figure 2.

Comparison of photographs of a real fabric with the simulation results.³⁹ (a) Real fabric: Unstretched (left) and worn (right) and (b) Simulated fabric: Unstretched (left) and worn (right).

In summary, geometric modeling is suitable for applications that require only quick computation and easy presentation. For applications that require physical properties and more rigorous calculation results, and can withstand longer calculation times, physical modeling is more appropriate. For the requirement to reflect the fabric structure, fiber texture, and other multi-realistic texture effects, yarn-structure modeling should be used, in which 3D single-yarn modeling is suitable for simple fabric structures and small area cloth modeling, while 3D yarn unit modeling can be suitable for more complex fabric structures and large area cloth modeling. Cloth modeling and simulation based on the yarn structure are computationally intensive and cannot yet achieve a good real-time visualization of clothing. However, this approach can reflect the fabric details and the appearance of the simulated fabric is realistic, which is especially important for virtual fitting. So, there is still considerable scope for useful in-depth research.

Multi-resolution grid simulation

Numerical integration is traditionally used in cloth simulation. Approaches include display integration, implicit integration, and Verlet integration. The method produces very limited improvements. Moreover, it is complex and time-consuming. So, to achieve a more realistic cloth simulation and minimize the computational effort, Zhang et al.³⁵ proposed a hierarchical multi-level grid method in 2001. The size of the triangular grid cells at each simulation stage is smaller than in the previous stage. The method was adapted for simulating overhanging cloth. In contrast, for a multi-resolution grid simulation, a fine grid is used in the complex areas where the cloth deforms, such as folds.⁴⁰ The rest of the grid is coarse, which to a certain extent reduces the computational volume. These approaches broaden the possibilities for cloth simulation. The main concern of these approaches is how to determine which areas of the cloth need to be divided into a fine grid and which parts can be divided into coarse grid.

Machine learning method to extract the degree of cloth bending and deformation

Some scholars have used machine learning methods to define a fine grid that can be used to determine the degree of bending and deformation of the cloth. Typical approaches that have obtained good results include the work by Min et al.⁴¹ in 2015 and Yanxia et al.⁴² in 2019. The former used a preprocessed high-precision cloth animation based on k-means clustering to extract bending deformation patterns. The multi-precision grid fabric geometry model is then refined, as shown in Figure 3. The number of vertices is significantly reduced, but the grid is finer at the pleats to maintain the complexity. This multi-precision grid cloth animation model also utilizes a force relation, etc. Although this method can improve the computational efficiency by about 70% while maintaining the realism of the animation, preprocessing and extracting the instance data are time-consuming. The method does not support dynamic grid simplification, and it is not effective when the movements of the target human are significantly different from the movements used for learning. In contrast, Yanxia et al.⁴² applied a quantum-behaved particle swarm optimization algorithm for bird foraging behavior to search for where the cloth bends, which can speed up the search for vertices to subdivide the cloth surface. Moreover, this cloth simulation has a mathematical model of air resistance, as shown in Figure 4, which makes the cloth movement more realistic. However, the edges of the cloth are irregular after falling.

Figure 3.

Two multi-resolution mesh skirt models with different numbers of vertices.⁴¹ (a) 6586 vertices and (b) 4056 vertices.

Figure 4.

Subdivision of the cloth model.⁴²

Integrating techniques from different research fields for zoning

In addition to using machine learning algorithms to identify where bending occurs, some scholars have integrated techniques from different research fields to select the partitions of a multi-resolution grid. In 2015, Koh et al.⁴³ proposed a dynamic adaptive multi-resolution grid for modeling cloth worn by humans. It is based on a given camera motion, which is used to adjust the criteria for controlling the refinement to ensure that the cloth model is dynamically generated for the line of sight in close conjunction with the human action. For scenes with multiple characters or large crowds, it can produce some computational savings. However, the method is still slow in reconstructing the grid after the cloth is deformed by human action, so it is not suitable for practical applications.

In 2021, Shi et al.⁴⁴ proposed using eye tracking to create a saliency prediction model. This was combined with a view grid reconstruction method to control the division of the adaptive grid. Figure 5 illustrates the saliency of 3D vertices. A triangle with high saliency in the figure is represented in red and one with low saliency in green. The back area has low saliency by default. The simulation effort of the method is also more considerable. This approach greatly reduces the number of vertices and surfaces in the clothing model, thus improving simulation efficiency. Monochrome scenes can be sped up by more than three times. This efficiency improvement is important for virtual fitting, film and television animation, etc.

Figure 5.

Significance of three-dimensional vertices.⁴⁴ (Color online only.) (a) Calculated heat map and (b) Red and green indicate high and low significance, respectively.

Reduce the amount of meshing with a hierarchy

To reduce the computational effort, some scholars have reduced the amount of meshing with a hierarchy. For example, In 2018, Wang et al.⁴⁵ proposed a nonlinear, adaptive cloth simulation method. This method starts by introducing a full multi-grid into a convergent nonlinear multi-grid, then they investigated the use of the adaptive smoothing approach to accelerate the simulation, and finally they also discussed how to deal with collision contact problems. The method requires a uniform distribution of grid vertices, but the actual situation is not always satisfied, and the handling of collision problems is not fully explored, resulting in a less than obvious improvement in simulation efficiency. Lee et al.⁴⁶ in 2019 proposed a simulation method that downsampled and downscaled the target cloth to get a miniature piece of cloth. Physical modeling of the miniature cloth retains the high-frequency details. An amplified depth neural network (DNN) is used to scale the miniature cloth to the target cloth for the final visualization, as shown in Figure 6. This method can reduce the computation time, but the micro cloth will have a significant impact on the simulation results if not selected properly. Moreover, it is difficult to make an accurate inference from data without training.

Figure 6.

Going from a micro-fabric to the target fabric.⁴⁶ DNN: depth neural network.

In summary, there are today many machine learning algorithms. Some use hybrid models instead of complex numerical integration algorithms. Cloth simulations based on machine learning methods have become popular in recent years. They can incorporate the environmental conditions and use techniques from different research fields and a refined grid for where the cloth deforms. These approaches promote the creative development of cloth simulation, and they will become the mainstream research methods.

However, difficulties remain. For example, after searching and extracting the cloth bending and deformation data, the refined part of a multi-resolution grid is difficult to change in real time. In addition, the search efficiency and re-gridding efficiency are low.

Relation between human posture and cloth deformation

To achieve realistic virtual fitting, it is not enough simply to simulate the cloth, as the human body model needs to “wear” the 3D clothing⁴⁷ and the clothing needs to respond to the movements of the human body. This would allow the visualization of the dynamic deformation of clothes. Therefore, not only the cloth draping characteristics,⁴⁸ yarn structure, etc., should be considered, but also other aspects, such as the movement of the human body. An example is the simulation of a crease in clothing caused by bending a leg. These need to be integrated into a virtual fitting simulation.⁴⁹ In this section, we focus on the progress of research into the relation between human posture and garment deformation in cloth simulation.

Modeling based on training from a cloth animation

Scholars have recognized that cloth folds depend on the wearer’s pose. Thus, the initial simulation methods were based on learning from a large number of training instances. For a given human pose, they predicted the cloth deformation based on neighboring instances. For example, in the method of Wang et al.⁵⁰ proposed in 2010, high-quality cloth folds are combined in ragged simulations that compute global and dynamic aspects of cloth motion. The method proposed by Xu et al.⁵¹ in 2014 generates a realistic animation for various clothes in real time. However, these works did not delve into the relation between human posture and garment deformation, but rather enriched the learning by processing a large number of example data points. The cloth folds were obtained by approximating target human movements to the examples.

On the other hand, in 2016, Deng et al.⁵² analyzed the relation between human posture and garment deformation in cloth animation, as shown in Figure 7. They first defined human pose features and the distribution of deformation features in the cloth grid. They extracted feature data from the simulation to obtain high-precision clothing animation examples. Then, they trained a neural network to learn the relation between human pose features and cloth deformation to derive a mature adaptive multi-precision clothing deformation model. However, this method has some limitations and the trained model cannot be adapted for other body types or clothing styles.

Figure 7.

Distribution of cloth deformation.⁵²

In 2017, Min et al.⁵³ also conducted a study on the relationship between the two. Firstly, a human motion database and cloth model were directly considered, where the cloth model was added to the human motion for generating the clothing animation and the deformation data of the cloth following the motion would be retained. Then the motion features and cloth deformation data were extracted by machine learning methods to construct a mathematical model of the relationship between the two for predicting the clothing deformation results under the new posture. They focused on comparing how four machine learning models learned the relation between the two, and the errors between the mathematical models of the two relations created separately. The errors for the four machine learning models are shown in Figure 8. The RF method (Random Forest) has the lowest average error but is slower due to its better accuracy. Moreover, it is difficult to achieve on-the-fly refreshing with a multi-precision model.

Figure 8.

Comparison of the average errors of mathematical models built by four machine learning methods.⁵³Note: RF, random forest; BP, back propagation neural network; GRNN, generalized regression neural network; SVM; support vector machine.

Scholars have studied the effect of human posture on predictions of cloth deformation. The aim is to reduce the calculation volume while improving the prediction efficiency with finer results. In addition, by learning the relationship between the two, the results of cloth deformation in a new posture can be better calculated, instead of relying only on the sample library to get approximate results, which has an important role in the development of clothing simulation. With the gradual development of various techniques, more types of machine learning methods are being explored for virtual fitting.

Modeling based on 2D image processing

Generative adversarial networks (GANs) were first proposed by Goodfellow et al.⁵⁴ Inspired by the original GANs, several variants have been applied to a wide range of fields.⁵⁵ In 2017, Jetchev et al. proposed a conditional analogy⁵⁶ GAN (CAGAN), which used GANs for virtual fitting.

In 2020, by extracting the shape of the human body from a photograph taken by a user, Hashmi et al.⁵⁷ were able to reduce the noise redundancy of local similarity and then pose estimation. GANs can learn and generate a garment selected by the user and map it to the human body using seam and point matching in neural body fitting to improve the accuracy of virtual fitting. However, the method ignores the dynamic aspects of the human body and garment mapping, and the effect of this is still to be investigated.

In 2021, Xiaochun et al.⁵⁸ applied an improved CAGAN, a mask network, and a deformation network together in virtual fitting based on the work of Jetchev et al. They quantitatively compared the results with those from the currently more advanced VITON and CP-VITON methods. It showed details of the clothes better and maintained the integrity of the body. It is especially good at dealing with limbs obscuring the body, as shown in Figure 9, but there are obvious deficiencies in the pleating and edge contours.

Figure 9.

Qualitative comparison of the three methods.⁵⁸

This method is based on 2D image processing. It has high computational efficiency, uses more relevant human poses, and provides a quantitative evaluation of the simulation results. However, the current mostly rough conversion of information, such as object classes and attributes, and the inability to generate cloth details and reflect physical characteristics are the main drawbacks of this method. In addition, the virtual fitting is limited to 2D visualization and is yet to be developed into three dimensions. The use of GANs in cloth simulation is still at the initial stage. With further research, it is believed that CAGANs will overcome some of these issues.

Collision and friction in cloth simulations

Cloth is a flexible fabric that is easy to deform. The dynamic nature of cloth results in collisions and friction with surrounding objects as well as with itself. Thus, collision and friction are some of the main problems in simulating cloth. Appropriately handling these features has an important role in the realism of cloth simulation and in reducing the simulation time.

In 2002, Bridson et al.⁵⁹ combined effective repulsion and a robust geometric treatment of collisions in an efficient algorithm for handling collisions, contact, and friction in cloth simulations. The resulting thickness model produced a realistic simulation. This classical Bridson model is widely used by researchers. However, the Bridson model has some limitations and cannot satisfy the cloth collision problem in all cases. Moreover, its computational efficiency also needs to be improved. Therefore, scholars have proposed various optimized models, solvers and so on in recent years.

In 2009, Selle et al.⁶⁰ proposed a history-based repulsive or collision framework that can be computed using the last known collision-free state. The frequency of the geometric collision computations is reduced and the efficiency at each time step has been improved. The method is best applied to cloth simulations with high-resolution requirements.

In 2013, Chen et al.⁶¹ used a three-layer model, with an outer layer of cloth, an inner layer with a deformable body, and an air layer in between. They proposed a fast and effective surface traversal algorithm for detecting collisions between the cloth and deformable objects, but it cannot handle self-collisions. Instead, it uses the continuous collision detection technique developed by Bridson et al.⁵⁹ It relies on friction measurement equipment to obtain friction data. In 2015, Ling et al.²³ used an AABB human surround box and human front and back depth images for real-time collision detection, and implemented a continuous collision response based on normal images and a modified DDA linear rasterization algorithm with fast pre-processing, but they also did not validate the self-collision problem. The approach developed by Hui et al.⁶² in 2018 is also unable to handle self-collisions. It is mainly focused on reducing the oversampling between cloth triangles due to co-edge cases during cloth boundary sampling. The sampling speed is fast.

In 2018, Tang et al.⁶³ proposed an incremental collision-processing algorithm for a GPU-based interactive cloth simulation that utilizes the spatial and temporal coherence of successive iterations of an optimization-based solver to compute the collision response. A somewhat larger time step can be chosen.

In 2019, Verschoor and Jalba⁶⁴ solved the collision contact problem by directly solving a hybrid linear complementary problem. They applied convex quadratic programming with linear constraints and used a conjugate residual solver as the backbone of the collision response system to ensure frictional accuracy. However, the conjugate residual solver results in a cloth model with weak stability.

In 2022, Chen et al.²⁴ proposed a nonlinear particle spring model in which all moving objects are particles. The improved collision detection method with an axis-aligned bounding box was then used for the fabric simulation. The real shape and data parameters of the cloth were obtained from a 3D image reconstruction, which was used to verify the simulation and collision processing. Figure 10 compares the effect of draping after the free release of a cloth from a horizontal position. The results were better, but the study focused on improving accuracy, so that the time consumption remains huge.

Figure 10.

Effects of draping: (a) photographs of real cloth items; (b) reconstructed three-dimensional images and (c) simulation results. Case 1: cotton fiber; Case 2: fibrilia; Case 3: polyester fiber; Case 4: 10% cotton and 90% polyester fiber.²⁴

A variety of collision and friction models have been proposed for cloth simulations. In particular, there has been a significant amount of research from different angles into the problem of collision detection and response, and great progress has been made. However, at present, the optimization of the friction model, real-time processing of the collision response, and the super-ejection phenomenon are still obvious shortcomings and need further research.

Analysis and discussion

From our in-depth research and analysis, we make the following insights into the research trends for key techniques in cloth modeling and simulation for virtual fitting.

Cloth modeling and simulation based on the yarn structure can better reflect the fabric structure and yarn characteristics and fully represent the appearance of the fabric. Thus, such methods have now become a hot topic. We believe that integrating the physical properties and structure of the fabric into the unit model to realize a 3D model of the yarn structure will be more likely to lead to a realistic simulation. However, there has been only a limited amount of current research in this area, so that further in-depth research is necessary.

In cloth modeling and simulation based on machine learning algorithms, techniques from different research fields can be integrated to select the partitions of a multi-resolution grid. For example, eye-tracking technology can indicate what the viewer is looking at and this can obviously be utilized to reduce the size of the grid and increase the simulation speed. Therefore, combining machine learning algorithms with techniques from different research fields has considerable possibilities for improving the effectiveness and speed of cloth simulation for virtual fitting.

To compare the advantages and disadvantages of different representative methods for cloth simulation and for modeling the relation between human posture and cloth deformation, the parameter values used by Wang et al.,⁵⁰ Xu et al.,⁵¹ Deng et al.,⁵² and Min et al.,⁵³ who trained human postures and clothing deformations using animated videos, are listed in Table 1. The RF method of Min et al. uses a larger training set but makes faster predictions, which illustrates that the overall simulation speed has improved significantly with the development of computer technology.

Table 1.

Comparison of parameters for different modeling approaches (with a T-shirt as an example)

Research methodology	Training sample size	Number of clothing vertices	Prediction speed /FPS
Wrinkle synthesis method of Wang et al.⁵⁰	527 action samples	23,000	11.65
Skinning method of Xu et al.⁵¹	150 action samples	6000	61
BP method of Deng et al.⁵²	21 joint dimensions, 8405 action samples	104 regional seed vertices	N
RF method of Min et al.⁵³	21 joint dimensions, 9000 action samples	14,362	651.34

Note: N, indicates no specific data are available; BP, back propagation neural network; RF; random forest.

The CAGAN method based on processing 2D images developed by Hashmi et al.⁵⁷ used a training set with 341,000 samples and a test set with 67,000 samples. Its prediction speed is faster. The average prediction accuracy and F₁ score were used to quantitatively evaluate the virtual fitting simulation effect of various types of clothing. This approach has obvious advantages over training and simulation processes that rely on high-precision animations.

Therefore, this paper argues that GANs have a greater potential and scope for further research into simulating a fabric for virtual fitting. The method should better incorporate fabric details and characteristics. GANs should use 3D reconstruction and other techniques to achieve 3D virtual fitting of clothing on the human body. We believe that with the continual progress of computer graphics, the method will make a breakthrough in this field.

4. When modeling collision and friction, computational speed and occasion adaptability are the two most important factors. We believe that continuous collision detection methods have wider adaptability and accuracy for dealing with collision and friction. Moreover, common envelope box methods (e.g., oriented bounding box, axis-aligned bounding box, and k-discrete oriented polytopes) are not refined enough for virtual fitting, especially for human body surface features, such as the female waist, hips, and chest. Further research is needed on the impact of human body collision with clothing and the resulting cloth deformation.

Conclusions

Cloth modeling and simulation are a key technique for virtual fitting, which has a wide range of application prospects for clothing design, video games, and virtual reality, and also has a potentially huge commercial value. This paper analyzes the advantages and problems of different methods for modeling and simulating cloth for virtual fitting. It conducts an in-depth analysis and comparison of yarn-structure modeling, multi-resolution grid simulation, and the relation between human posture and cloth deformation. It investigates the collision and friction problem. This paper may serve as a new reference for the research and development of cloth modeling and simulation techniques for virtual fitting.

In summary, we conclude the following.

Machine learning algorithms have been extensively studied and have achieved good results for cloth simulation modeling for virtual fitting. Machine learning algorithms can be integrated with techniques from different research fields for cloth multi-resolution grid partitioning, which may lead to a breakthrough in cloth simulation. Therefore, we expect that cloth simulation methods will be enhanced to meet the future needs of real-time refined cloth simulations for virtual fitting with better realism.

There is no unified or feasible evaluation standard or index for evaluating the realism of a cloth simulation. This is limiting the development of virtual fitting and cloth modeling and simulation to a certain extent. Therefore, a future research direction in cloth modeling and simulation for virtual fitting is to develop such a metric.

Footnotes

Declaration of conflicting interests

The author(s) declare no conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This work was supported by the National Key R & D Program Fund of China (grant 2018YFB1308801).

ORCID iDs

Xingxing Li

Xin rong Li

Yuzhuo Li

Wenqian Feng

References

Wang

Shao

, et al. Learning an intrinsic garment space for interactive authoring of garment animation. ACM Trans Graphics 2019; 38: Article No.220.

Miell

Gill

Vazquez

Enabling the digital fashion consumer through fit and sizing technology. J Global Fashion Marketing 2018; 9: 9–23.

Hong

Bruniaux

Zeng

, et al. Design and evaluation of personalized garment block for atypical morphology using the knowledge-supported virtual simulation method. Text Res J 2018; 88: 1721–1734.

Zhiwei

Yuanyuan

Xin

, et al. Accurate retrieval of multi-scale clothing images based on multi-feature fusion. Chin J Comput 2020; 43: 740–754.

Han

Zhe

, et al. VITON: an image-based virtual try-on network. In: Helen (ed.) IEEE/CVF conference on computer vision and pattern recognition (CVPR), 18–23 June 2018, Salt Lake City, UT, New York, USA, 00787: 7543–7552.

Yong

Kai-Li

Yu-Han

, et al. Real-time collision detection between variform cloth in simulation and rigid bodies. J Chin Comput Syst 2016; 37: 2347–2350.

Liang

Shuguang

Lijuan

, et al. History and research hot spot of three-dimensional virtual garment display. J Text Res 2011; 32: 153–160.

Min

Tianlu

Shihong

, et al. Advances and challenges in cloth animation. Chin J Comput 2012; 35: 2446–2458.

Qinting

Yanmei

Xiang

Visualization simulation technology of cloth deformation based on mass-spring model and its prospect. J Text Res 2013; 34: 147–153.

10.

Mozafary

Payvandy

Study and comparison techniques in fabric simulation using mass spring model. Int J Clothing Sci Technol 2016; 28: 634–689.

11.

Meng

Suying

Wen

, et al. Design and development of clothing based on virtual fitting technology. Wool Text J 2020; 48: 4.

12.

Aijing

Jie

Virtual fitting system key technology research. Prog Text Sci Technol 2020; 3: 28–32.

13.

Nourbakhsh

Tian

, et al. A generic method of wearable items virtual try-on. Text Res J 2020; 90: 2161–2174.

14.

Yanxia

Jinrui

Yao

, et al. Progress of cloth simulation modeling. J Image Graphics 2021; 26: 970–977.

15.

Xiaohui

Review of digital pattern-making technology in garment production. J Text Res 2022; 43: 203–209.

16.

Ruomei

Xiaonan

Realistic cloth simulation based on geometrical measurement and deformation. J Softw 2015; 26: 1785–1799.

17.

Chen

Jiang

, et al. Three-dimensional simulation of metallic two-bar warp-knitted mesh based on loop structure geometry. J Text Inst 2016; 108: 1–10.

18.

Sha

Gaoming

Research status and development trend of 3-D simulation technology for weft knitted fabric. J Text Res 2016; 37: 166–172.

19.

Xin

Laihu

Minglai

, et al. Modelling and algorithm of weft knitted fabric. J Text Res 2018; 39: 44–49.

20.

Narain

Samii

O’Brien

JF.

Adaptive anisotropic remeshing for cloth simulation. ACM Trans Graphics 2012; 31: Article No.152.

21.

Tamstorf

Jones

Mccormick

SF.

Smoothed aggregation multigrid for cloth simulation. ACM Trans Graphics 2015; 34: Article No.245.

22.

Bowen

Ping

Yunye

3-D virtual try-on technique based on dynamic feature of body postures. J Text Res 2021; 42: 144–149.

23.

Ling

Mingmin

Zhigeng

, et al. Image-based detection and response of continuous fast collision between cloth and human body. J Softw 2015; 26: 1–7.

24.

Chen

Wang

Zhang

Accurate simulation of draped fabric sheets with nonlinear modeling. Text Res J 2022; 92: 539–560.

25.

Wang

. Study on improving three-dimensional dynamic simulation model and equation of flexible fabric. Adv Mech Eng 2019; 11: 1–6.

26.

Qiu-Feng

Jin

Research on dynamic effect of cloth in random wind field based on noise function. Comput Simul 2015; 32: 192–196.

27.

Provot

. Deformation constraints in a mass-spring model to describe rigid cloth behavior. Graph Interface 1995; 86: 95–147.

28.

Penggao

Gang

Ran

, et al. Fabric dynamic simulation by isogeometric mass-spring model. J Comput Aid Des Comput Graphics 2019; 31: 911–918.

29.

Wang

Deng

Xin

, et al. A novel three-dimensional surface reconstruction method for the complex fabrics based on the MVS. Opt Laser Technol 2020; 131: 106415.

30.

Mozafary

Payvandy

A novel method based on loop shape for simulating knitted fabric using mass spring model. Fibers Polym 2017; 18: 533–541.

31.

Chu

. A framework for extracting cloth descriptors from the underlying yarn structure. PhD thesis, University of California, Berkeley. 2005.

32.

Kaldor

James

Marschner

Simulating knitted cloth at the yarn level. ACM Trans Graphics 2008; 27: 65.

33.

Zheng

Study on general geometrical modeling of single yarn with 3D twist effect. Text Res J 2010; 80: 867–879.

34.

Peng

Jiang

Xia

, et al. Deformation and geometric modeling in three-dimensional simulation of fancy weft-knitted fabric. Text Res J 2020; 90: 1527–1536.

35.

Zhang

Sha

, et al. Warp-knitted fabrics simulation using cardinal spline and recursive rotation frame. J Eng Fibers Fabr 2017; 12: 29–38.

36.

Jiang

Tang

, et al. Three-dimensional simulation of warp knitted structures based on geometric unit cell of loop yarns. Text Res J 2020; 90: 2639–2647.

37.

Dong-Yong

Zhong

Li-Zhuang

. Fiber-based 3D cloth modeling and garment simulation with photorealistic appearance. J Softw 2016; 27: 2587–2599.

38.

Ying

Tiantian

Xiang

, et al. Micro-geometry modeling of three-dimensional orthogonal woven fabrics based on digital element approach. J Text Res 2020; 41: 59–66.

39.

Grechukhin

Rudovskiy

Sokova

, et al. Carbon fabric 3D modeling according to nonlinear bending theory. J Text Inst 2020; 111: 1511–1517.

40.

Litao

Jin

Sui

, et al. Multi-resolution cloth construction based on low-resolution cloth sampling. Appl Res Comput 2019; 36: 284–289.

41.

Min

Tian-Lu

Zhao-Qi

Multi-resolution cloth based on animation instances. Chin J Comput 2015; 38: 1296–1306.

42.

Yanxia

Siyue

, et al. Adaptive cloth simulation based on QPSO algorithm. Comput Eng Appl 2019; 55: 154–160.

43.

Koh

Narain

O’Brien

JF.

View-dependent adaptive cloth simulation with buckling compensation. IEEE Trans Vis Comput Graphics 2015; 21: 1138–1145.

44.

Shi

Ming

Liu

, et al. Saliency-dependent adaptive remeshing for cloth simulation. Text Res J 2021; 91: 480–495.

45.

Wang

Fratarcangeli

, et al. Parallel multigrid for nonlinear cloth simulation. Comput Graphics Forum 2018; 37: 131–141.

46.

Lee

IK.

Efficient cloth simulation using miniature cloth and upscaling deep neural networks. ACM Trans Graphics 2019; arXiv preprint arXiv:1907.03953.

47.

Cohen

Virtual reconstruction of 3D articulated human shapes applied to garment try-on in a virtual fitting room. Multimedia Tools Appl 2022; 17: 11071–11085.

48.

Petrak

Mahnić Naglić

Rogale

, et al. Analysis of polygonal computer model parameters and influence on fabric drape simulation. Materials 2021; 14: 6259.

49.

ESL

Aslam

, et al. A new method to evaluate the dynamic air gap thickness and garment sliding of virtual clothes during walking. Text Res J 2019; 89: 4148–4161.

50.

Wang

Hecht

Ramamoorthi

, et al. Example-based wrinkle synthesis for clothing animation. ACM Trans Graphics 2010; 29: Article No 107.

51.

Umentani

Chao

, et al. Sensitivity-optimized rigging for example-based real-time clothing synthesis. ACM Trans Graphics 2014; 33: Article No 107.

52.

Deng

Mao

Min

, et al. Cloth deformation prediction based on human motion. In: Helen (ed) IEEE International conference on virtual reality and visualization (ICVRV) – Hangzhou, China, 24–26 September, 2016, New York, USA, 49: 258–263.

53.

Min

Liu

Tianlu

, et al. Study on the correlation between human motion and garment deformation in cloth animation. J Comput Aid Des Comput Graphics 2017; 29: 1941–1951.

54.

Goodfellow I, Pouget-abadie J, Mirza M, et al. Generative adversarial nets. In: Z.Ghahramani (ed) Advances in Neural Information Processing Systems(NIPS)-Montreal, Canada, 8–13 December, 2014, San Francisco, USA, 27: 2672–2680.

55.

Cai

Zhao

Ban

, et al. GAN-based image-to-friction generation for tactile simulation of fabric material. Comput Graphics 2022; 102: 460–473.

56.

Jetchev N, Bergmann U. The conditional analogy gan swapping fashion articles on people images. In: Helen (ed) IEEE International Conference on Computer Vision Workshops (ICCVW)-Venice, Italy 22–29 October 2017, New York, USA, 269: 2287–2292.

57.

Hashmi

Ashish

Keskar

, et al. FashionFit: analysis of mapping 3D pose and neural body fit for custom virtual try-on. IEEE Acc 2020; 8: 91603–91615.

58.

Xiaochun

Hongwei

Chengfeng

Application of improved CAGAN in virtual try-on. Comput Eng Appl 2021; 57: 152–158.

59.

Bridson

Fedkiw

Anderson

Robust treatment of collisions, contact and friction for cloth animation. ACM Trans Graphics 2002; 21: 594–603.

60.

Selle

Irving

, et al. Robust high-resolution cloth using parallelism, history-based collisions, and accurate friction. IEEE Trans Vis Comput Graphics 2009; 15: 339–350.

61.

Chen

Feng

Wang

Modeling friction and air effects between cloth and deformable bodies. ACM Trans Graphics 2013; 32: Article No. 88.

62.

Hui

Zhen

Yanjie

, et al. Real-time collision detection method for fluid and cloth. J Comput Aid Des Comput Graphics 2018; 30: 602–610.

63.

Tang

Liu

Tong

, et al. I-Cloth: incremental collision handling for GPU-based interactive cloth simulation. ACM Trans Graphics 2018; 37: Article No. 204.

64.

Verschoor

Jalba

AC.

Efficient and accurate collision response for elastically deformable models. ACM Trans Graphics 2019; 38: Article No. 17.