Uses and Users of Confidential Commodity Flow Survey Data in the United States

Qualitative Approach

A qualitative approach was used to extract and analyze the substance of the proposals. We divided the 44 proposals roughly evenly between two of this paper’s authors, each of whom read and summarized their respective works. They identified the proposals’ focal research topics, mentions of CFS keywords, and variables used for study; additionally, they identified the context for the use of CFS data and limitations that CFS data posed to those who submitted the proposals. They noted methods that were planned to be used, highlighting those that were interesting or novel. Proposals that mentioned the high importance of CFS data for certain techniques were recorded. The professional associations of the proposals’ submitters were also captured. These findings were then consolidated into a single document.

Quantitative Approach

A quantitative approach was used to complement the qualitative approach and text analysis techniques were applied. Text analysis is a method to quantify the frequency of text, such as words found in a document. “Text” is considered as unstructured data and often contain information (e.g., words) not relevant to the study of interest. A series of preprocessing steps was performed to convert unstructured text data into structured text data formats suitable for analysis. To quantify text data, “term frequency” was used, defined as the number of times a term appears within a document. The calculation involves simply counting the frequency of each term in the document. The overall flow of text analysis process is visually summarized in Figure 1, and R software was used to process and analyze the text data.

OPEN IN VIEWER

Figure 1.

Workflow of text analysis for project metadata and proposals.

As for the 44 project proposals in PDF format, an R script was developed to detect the .pdf extension of all files in the working directory and create a vector storing the file names. Text extractions were then performed with the vector created as input. Specifically, the pdf_text() function in R was used to render all textboxes on a text canvas and returned a character vector of a length corresponding to the number of pages in the PDF file ( 11 ). The length in pages of each file was also extracted. The texts extracted from all the 44 files were stored in a single data object, with an indication of the source file. As for the spreadsheet file storing the metadata of the 849 project proposals, the text from the title and abstract columns was extracted and stored in separate datasets.

Preprocessing the text involved several tasks, including removing punctuations, numbers, and whitespaces, converting the text to lowercase, and applying stemming and lemmatizing to the words. Removing specific types of character and converting to lowercase are self-explanatory. “Stop words” are terms that do not contribute significantly to the semantics of the content and are typically removed during preprocessing. Removing stop words helps reduce the size of the dataset, which not only improves processing efficiency but also shortens training time ( 12 ). Custom stop words that were deemed irrelevant for the analysis and removed included “can,”“use,”“using,”“used,” and “also,” as well as quotation marks, dashes, and other words not related to the subject of focus from the extracted texts. Lemmatization relies on accurately determining the intended part-of-speech and the meaning of a word based on its context to reduce the word to its root form. For example, a lemmatization algorithm can map “runs,”“running,” and “ran” to the root form (i.e., lemma) of “run” ( 13 ).

Stemming is a technique used to reduce inflected or derived words to their base or root form, known as the “word stem.” This process helps to normalize words and group together different forms of the same word, making text analysis more efficient by treating variations of a word as a single entity. For example, the words “programming,”“programmer,” and “programs” all can be reduced to the common word stem “program.” Stemming algorithms work by identifying and removing common prefixes and suffixes found in inflected or derived words and it uses a list of frequent affixes to truncate the end or beginning of the word, reducing it to its root form ( 13 ). It should be noted that stemming was initially attempted and resulted in undesirable outputs. For example, stemming “productivity” and “products” resulted in the same word “product” and the meaning of “productivity” was lost. As a result, stemming was not applied for this analysis, and consequently, is not shown in Figure 1.

For the analysis of texts from the 44 proposals, additional preprocessing was performed to remove infrequently occurring words, using a sparsity parameter in which the analyst specifies the proportion of documents in which a term must appear to be retained ( 14 ). “Sparsity” indicates how often certain terms appear in the documents. For example, the sparsity parameter of 0.90 retains terms that appear in at least 90% of the documents, while terms that appear in fewer than 90% of the documents are removed. This parameter is determined based on the analyst’s judgement considering the study purpose and needs.

In addition to removing stop words, additional words were removed based on preliminary term frequency analysis. Those words, called “generic terms” in this study, include “year,”“paper,”“ et al.,” and “different,” as few examples. Identifying generic terms for removal is an iterative process and depends on the field of the study. In addition, the number of words to be considered together is also an important consideration. Thus, identifying and removing generic terms were performed separately for one-word, two-word, and three-word terms.

Term frequency analysis was conducted by counting and displaying the most frequently appearing terms in a bar chart. A term can consist of more than one word. This study explored the number of words for a term and limited it to three words for succinctness and created bar charts, called “N-gram” (i.e., unigram) for one-word terms, “bigram” for two-word terms, and “trigram” for three-word terms. This analysis reveals focuses of text based how frequently each term appears across the documents.

Topic modeling was conducted to identify underlying topics or themes using natural language processing (NLP) and text mining methods. Topic modeling leverages computing power and automation to systematically explore and analyze large volumes of text data, enabling identification of underlying themes ( 15 ). It was applied on the extracted words to discover topics characterizing the projects in the project metadata and proposals. Among many topic models, the most frequently used one, latent Dirichlet allocation (LDA), was used. LDA is a Bayesian hierarchical probability generation model capturing similarities across different data components. Each data component consists of a document, and the collection of documents is referred to as a corpus. A document is considered as a mixture of topics, where each topic is a probability distribution over a set of terms; in this framework, each document itself is a probability distribution over a set of topics ( 12 ). Topics are identified when similar sets of words consistently occur together across multiple documents. These recurring word patterns help to cluster related terms and group them into distinct topics.

The number of topics needs to be determined before estimating a topic model and there are various ways to determine the number of topics ( 16 ). It should be noted that the true number of topics is unknown. It can be determined by an analyst based on their expert judgement and/or a quantitative method. This study employed both methods—a quantitative method on the project metadata, specifically the harmonic mean technique, and an expert judgement in conjunction with a quantitative method on the project proposals. For topic modeling of the project proposals, the most appropriate topic for the given number of topics was chosen based on coherence. Coherence score measures how well a topic is supported by text. A higher value of coherence indicates that the LDA model is performing well, with specific topics representing the text being analyzed more accurately. This suggests that the terms within each topic are more semantically related, making the topic more meaningful and interpretable. The UMass coherence score, commonly used for evaluating topic models, was calculated to assess the quality of the topics generated by the LDA model ( 17 ). This score helps determine how well the top terms within a topic are related to each other, with a higher score indicating better coherence and more meaningful topics.

It is important to note that topic models, such as LDA, are multi-membership models, meaning that a single document can be associated with multiple topics or themes. This characteristic requires a sufficient volume of text to properly develop a topic model. While there is no universally established threshold for the minimum word count in topic modeling, it is generally recommended that a corpus contain at least a few thousand words, with each individual document consisting of several hundred words, to ensure meaningful topic extraction.

Submitters

Most of the submitters listed in the proposals were associated with universities. Some submitters specified their academic association with graduate-level business schools within universities. Other submitters named themselves as members of the National Science Foundation or the Federal Reserve Board. Finally, a few submitters cited their associations with the National Bureau of Economic Research and the U.S. Department of Agriculture. The number of submitters noted in a proposal ranges from one to nine, with a median of three submitters, as shown in Figure 2.

OPEN IN VIEWER

Figure 2.

Number of participating submitters in the 44 project proposals requesting confidential Commodity Flow Survey data (2011–2022).

The submitters looking to access confidential CFS data were well-credentialed and generally worked in economic research. Academic economic research appears to receive strong interest. It may be possible that less-credentialed researchers faced roadblocks to submitting proposals for accessing confidential CFS data. More research would be necessary to consider this issue, but the priority for confidential CFS data is to maintain their federal confidentiality requirements; it is expected that such data protections prevent full, unfettered accessibility that would be ideal for less sensitive data products.

Searching for articles and papers at Transport Research International Documentation (TRID) with a keyword of “commodity flow survey” limited to articles, papers, reports, books, collections, and records with abstracts brought up 378 entries (as of April 29, 2024). TRID is an integrated database combining the records from TRB’s Transportation Research Information Services (TRIS) database and Organisation for Economic Co-operation and Development (OECD)’s Joint Transportation Research Centre’s International Transport Research Documentation (ITRD) database ( 18 ). Many of these entries were papers from transportation-related journals such as Transportation Research Record, Transport Policy, and Research in Transportation Business and Management. Many of them appear to have used publicly available CFS data and tables. Among the 378 entries, the vast majority (363) included “freight” in their records.

Research Areas

Every proposal except one involved economic research. The specific areas of research can be roughly divided into four categories: business practices, market phenomena, high-level trends, and synthesis approaches. A few proposals incorporated work in the field of public health in association with pandemics and pollution. Table 2 summarizes research areas in the four categories.

OPEN IN VIEWER

Table 2.

Categorization of Research Areas Identified in 44 Project Proposals Requesting Confidential Commodity Flow Survey Data

Category	Definition	Research areas
Business practices	Describing structures that exist at the firm/establishment level; examining business decisions that occur at the firm/establishment level	Agglomeration, co-location, firm organization, globalization and offshoring, management practices, research and development, technology adoption
Market phenomena	Analyzing markets	Collusion, competition, labor markets, price indices, trade and trade effects, wages
High-level trends	Studying the overarching changes/forces in society and the economy	Automation, economic growth, economic policy, firm heterogeneity, inequality, innovation, productivity
Synthesis approaches	Compositing multiple levels of analysis, (e.g., both business practices and high-level trends)	Asset reallocation, local effects, responses to shocks, supply chain networks

It should be noted that these four categories of research area were identified as the human reviews progressed and correspond to key topics within each proposal, rather than to the individual proposals themselves. Each proposal generally addresses multiple topics, which may span across more than one category in the table. For example, many proposals include topics that are classified under the category “Synthesis approaches.”

Data Elements

Intended use of specific CFS variables was mentioned in the proposals and is summarized in Figure 3. Specifically, geographic variables, such as origin and destination of shipments, were the most used, in 31 proposals, while shipment value was used the second most, in 21 proposals. The export and commodity (Standard Classification of Transported Goods [SCTG]) variables were used by 12 proposals, respectively, shipment weight used by nine proposals, shipping distance and import variables by four proposals each, and shipping mode and industry (North American Industry Classification System [NAICS]) variables by three proposals each.

OPEN IN VIEWER

Figure 3.

Bar plot of Commodity Flow Survey (CFS) variables found in the 44 project proposals requesting confidential CFS data (2011–2022).

Clearly, the geographic information provided by CFS is useful to data users, suggesting that even more geographic data may be considered for provision to users. The geographic information is used not only for analysis but also for data linkage across different datasets. It is unsurprising that proposals which primarily focused on economic aspects place greater emphasis on shipment value rather than shipment weight. Similarly, economic researchers favored export research and commodity analysis.

Shipping mode is a key variable for transportation practitioners and researchers, yet it has garnered limited attention in research utilizing confidential CFS data. Given that the majority of the proposals focus on economic research, it is somewhat surprising that NAICS did not receive higher attention, as it is closely associated with economic analyses. However, it is possible that NAICS information is available in other datasets requested alongside CFS data, potentially explaining the relative lack of interest in it in these proposals.

Feedback

CFS data were found to be highly important to a majority of the proposals. Most feedback was of a more general nature: each proposal described the benefits to the U.S. Census Bureau that follow the proposed research and, in these sections, submitters often noted small flaws or data gaps in existing census data. However, 38 of the 44 proposals described how CFS data would be used, and the data were characterized as indispensable for many analysis subjects, such as firm-level trade, supply chain, market size, changes in firm shipping because of shocks, geography of firm sales, cross-industry connections, linkages between regions and industries, transportation because of firm shipping, and shipping mode choice.

Several proposals incorporated explicit feedback about CFS data. Specifically, data users requested variables such as fuel price, shipping speed and cost, weight-to-value ratio, and destination firm. Finally, several proposals raised the issue that agriculture analysis is made difficult by the CFS scope design, because livestock and food shipped from farms are outside its scope.

This explicit feedback primarily reflects demands for more data, a common request across most data collection programs. Meeting these demands is challenging, mainly because the resources allocated for data collection have remained relatively constant over time for most data programs. However, by leveraging administrative records and technological advancements, it may be possible to collect more data and more detailed data without exceeding the current resource constraints.

Adding more data involves two key aspects: increasing shipment records and including additional data elements in the data collection. For CFS, more shipment records could be achieved through the cooperation of shippers who maintain shipment records in databases, along with the system capabilities to collect and process large volumes of data stored in various formats. For 2022 CFS, BTS and the U.S. Census Bureau received an entire shipment log for limited reporting periods, rather than sampled records, from certain shippers. The 2022 CFS collected about 100 million initial unvalidated records, compared with about 6 million final validated records in 2017. The final number of validated records for the 2022 CFS is expected to be approximately 83 million. The 2022 CFS final tables (CEDSCI and visual) were released on June 26, 2025, and the 2022 CFS PUF is scheduled for release in Fall 2025 ( 19 ).

For future data collection, ongoing efforts should focus on encouraging more shippers to provide complete shipment logs, while also exploring methods to capture key data elements without adding to their reporting burden. For instance, during the 2022 CFS, several large firms took advantage of the bulk data submission option, as they typically maintain shipment records in database systems. In contrast, smaller businesses often keep invoices and shipment logs in less structured formats, such as PDFs or paper documents. Leveraging intelligent document processing tools that combine artificial intelligence with optical character recognition can help transform these unstructured formats into machine-readable text, which can then be extracted and stored in databases in usable formats for CFS.

Collecting additional data element is another key demand for more data. Shipping cost, as highlighted in the feedback of the qualitative review of this study and through feedback channels for public CFS data, such as the Innovations in Freight Data Workshops, is an important and useful piece of information ( 8 ). This is being explored for inclusion in future data collection efforts.

It should be noted that adding a question to the CFS questionnaire involves significant preparation, coordination, and approval. One of the considerations is estimating the time a respondent would spend answering the additional question, including the time required to retrieve and maintain information. Adding a question increases the total time burden on respondents, and this increase must be justified in the submission of the information collection request to OMB. Another consideration is conducting cognitive interviews for the new question to ensure that respondents understand and answer the question as intended. These interviews help identify potential issues with question clarity and interpretation, ensuring that the data collected is valid and reliable. In summary, while adding a question may seem straightforward, the process of integrating it into the survey questionnaire is more complex and time-consuming than it appears.

The feedback provided in the proposals offers new insights into the needs of data users, and also confirms data gaps that we had previously recognized through public channels. It aligns, to some extent, with the demand for additional data that has been recognized by users of public CFS data. The types of analysis for which CFS is essential should be further researched, and any future changes to CFS should aim to facilitate these analyses. Some of the desired information is challenging to collect because of the nature of CFS. For instance, agriculture and international trade are out of the scope of CFS. To complement CFS in these areas, FAF was developed by collecting and integrating additional data into CFS.

Metadata of 849 Project Proposals

Metadata of the 849 proposals, referred to in this paper as the “849 metadata entries,” were analyzed by term frequency and topic modeling, focusing specifically on two elements: the title and the abstract ( 10 ).

Term Frequency Analysis

Titles and abstracts from the metadata were examined to gain a general understanding of the focus of the 849 metadata entries. Three separate analyses were performed by the number of words in a term: one-word term, two-word term, and three-word term. In addition to general stop words, generic terms were further removed to uncover popular terms reflecting the characteristics of the proposals.

Frequently found one-word terms appearing at least 20 times in the titles and at least 200 times in the abstracts are summarized. The term “firm” comes at the top, 155 times, followed by “market” (91), “labor” (90), “business” (53), “productivity” (51), and so on, in the titles. As for the terms in the abstracts, “firm” is the most found word (1,695 occurrences), followed by “market” (599), “business” (533), “economic” (509), “establishment” (493), and “labor” (490). The terms appear to be related mostly to business, economy, and labor, such as firm, market, employment, trade, and worker, but to cover a range of subjects such as house, health, innovation, social, and residential. The top two one-word terms are the same in both the titles and abstracts. However, it is difficult to understand the focuses of the proposals based on a single word. For example, “house” could be an economic issue when a term “market” is attached while could be a social issue when “low-income” is attached.

To develop better insights from the titles and the abstracts, terms composing two adjacent words (i.e., two-word terms) were examined, and those from abstracts are visualized in the bigram in Figure 4. The results provide more clear understanding of the proposals. “Labor market” comes top and “firm level,”“health insurance,”“establishment level,”“plant level,” and “household dynamic” follow. The terms are closely related to business and economic aspects, although some distinctive terms such as “residential segregation,”“criminal justice,” and “information technology” also emerge. Consecutive three words (i.e., three-word terms) were also examined in the title and abstracts. “Labor market outcome” appeared the most, with 18 occurrences in the titles, and 44 occurrences in the abstract.

OPEN IN VIEWER

Figure 4.

Bigram of two-word terms in abstracts of the metadata of the 849 project proposals requesting confidential data (1998–2023).

Figure 5 presents the two-word and three-word terms as a word cloud, with a different number of terms included—the top 100 terms are shown in the cloud. “Firm” and “labor market” stand out as the top one-word and two-word terms. In the two-word cloud, the top terms appear alternately above and below “labor market,” generally matching the bigram (Figure 4). Although the clouds visually appear informative, a key caveat is the absence of quantitative information (word frequency).

OPEN IN VIEWER

Figure 5.

Word cloud of one-word and two-word terms in abstracts of the metadata of the 849 project proposals requesting confidential data (1998–2023): one-word terms (left) and two-word terms (right).

The 849 metadata entries cover various subjects yet mostly explore issues in business, economy, and labor market. Two-word terms appear to provide clearer insights into project characteristics than either one-word or three-word terms.

Topic Modeling

To discover latent topics of the 849 metadata entries, topic modeling was performed using an LDA model. However, typical topic models such as LDA are not well-suited for analyzing short text because they assume each document can belong to multiple topics, which are considered as multi-membership models. In contrast, the Short Text LDA with Clustering model, which assumes that each document belongs to only one topic, is better suited for analyzing short text and could be more effective for this type of data ( 20 ). Although there is no established threshold for the minimum number of terms required for topic modeling, it is generally recommended that the corpus includes at least a few thousand terms, with each individual document containing few hundred terms. For the titles of the 849 metadata entries, the average term count is 10, ranging from 2 to 31. This low term count in each document limits the effectiveness of LDA topic model, making it unsuitable for this context.

On the other hand, the abstracts contain an average of 200 terms, ranging from 15 to 1,042, and were deemed acceptable for topic modeling. The initial corpus, constructed from the abstracts, contains 172,718 tokenized terms, indicating the abstracts are suitable for LDA topic modeling. It should be noted that six proposals had no text in the abstract column of the metadata file, meaning a total of 843 abstracts were used for topic modeling.

The number of topics was determined based on the harmonic mean method ( 21 ). To find the optimal number of topics for the corpus, a sequence of LDA models was estimated based on Gibbs sampling by ranging the number of topics from 2 to 100 by an increment of 1; Gibbs sampling tends to produce more coherent and interpretable topics ( 22 ). The number of topics (n = 47) corresponding to the highest harmonic mean of the log-likelihood values of the models was determined to be optimal. This suggests existence of diverse themes among the 843 abstracts.

Top five terms were extracted to express the 47 latent topics, which are plotted in Figure 6. The topics show diverse issues ranging from technology, health, and diversity to education, energy, and immigration. The most frequent topic is associated with trade, including export and import, followed by topics associated with innovation and technology, health care, and ethnic issues.

OPEN IN VIEWER

Figure 6.

Top 25 five-term topics generated from the abstracts of the 843 proposals requesting confidential data (1998–2023).

To effectively visualize and interpret topics generated by LDA topic modeling, LDAvis was used to create Figure 7 ( 23 ).

OPEN IN VIEWER

Figure 7.

Visualization of the 47 topics extracted from the abstracts of the 843 proposals requesting confidential data (1998–2023): scatter plot (47 topics) (left) and bar plot (Topic 7) (right).

LDAvis employs principal component analysis to reduce the dimensions of the topic-term matrix and to display the topic model results in an interpretable fashion. The scatter plot (Figure 7 left) shows the distribution of 47 topics in a principal component space, with PC1 and PC2 representing the first and the second principal components of the topic-term matrix, respectively. The plot, called the “inter-topic distance map,” illustrates the relationship among topics by visualizing their relative distances after multidimensional scaling ( 24 ). The size of each circle is proportional to the frequency of topics, meaning that a larger circle corresponds to a more prevalent topic in the corpus.

The bar plot (Figure 7 right) displays the 30 most relevant terms for the selected topic. Terms are ranked by their relevance scores, which make each topic more distinctive and aid interpretation. The relevance score is not a direct count but reflects a term’s frequency within the topic adjusted for its overall frequency in the corpus (see Sievert and Shirley for technical details) ( 23 ). As a result, a term may occur frequently yet still have a low relevance score; a high relevance score indicates that a term strongly represents the topic. Each term is shown with two bars: dark gray for terms highly relevant to the selected topic and light gray for terms common overall but less distinctive. This plot helps convey the semantic meaning of the topic.

The distance between circles on the scatter plot (Figure 7 left) represents the semantic relationship between the corresponding topics. Topics whose circles are close to each other are more similar, while those farther apart are less related. Three visually distinct groups of topic are observed. Topics 1 and 7 are notably distant from each other as well as the remaining 45 topics. Topic 1 is associated with terms “health – insurance – care – meps – premium,” which can be labeled as “Health Insurance and Care,” indicating a focus on healthcare access and financing. The term “meps” refers to the Medical Expenditure Panel Survey, a dataset that collects detailed information on the cost, utilization, and coverage of health care and health insurance ( 25 ).

Topic 7 corresponds to terms “trade – export – import – foreign – lftld,” which can be labeled as “Trade and Foreign Export-Import.” This topic is closely related to CFS data. This suggests that research associated with CFS is unique among all the projects requesting access to data through FSRDCs. The bar plot (Figure 7 right) presents the term frequencies specific to Topic 7, highlighting its distinct nature. The ratio of the length of the dark bar to that of the light bar indicates the prevalence of each term. For example, the bar for “trade” demonstrates that the term is predominantly associated with Topic 7, appearing approximately 98% of the time within the topic. A similar, though to a lesser degree, predominance is observed for other top terms such as “export” and “import.” It should be noted that the bar plot was created with the relevance parameter set at 1.0, which ranks terms within a specific topic ( 23 ). Changing the relevance to any value within the range of 0 to 1 did not alter the top three terms. Twenty-five of the 47 topics are displayed in Figure 6, with each topic represented by its top five terms.

The top five datasets most frequently requested in proposals that included CFS were: 1) the Census of Manufactures, 2) the Annual Survey of Manufactures, 3) the Standard Statistical Establishment Listing, 4) the Longitudinal Business Database—No Revenue, and 5) the Longitudinal Foreign Trade Transactions Database. Meanwhile, the top five datasets requested in proposals that did not include CFS were: 1) the Census of Manufactures, 2) the Standard Statistical Establishment Listing, 3) the Annual Survey of Manufactures, 4) the Longitudinal Business Database—No Revenue, and 5) the American Community Survey. Notably, four out of the five datasets were common to both groups. This overlap suggests that, while the types of data requested are largely similar, the specific research focus may differ between proposals linked to CFS and those not. The proposals that included CFS requested an average of 16 datasets (including CFS), whereas those that did not include CFS requested an average of eight datasets. This difference suggests that research using CFS data may be inherently more complex, often necessitating a wider range of datasets to conduct a comprehensive analysis than research not using it.

44 Project Proposals Requesting CFS Data

Texts extracted from the 44 proposals that requested access to confidential CFS data were analyzed. A total of 13,188 terms were identified after removing stop words, generic terms (e.g., prepositions, conjunctions, and pronouns), and punctuation. One-word and two-word analysis of term frequency was performed for the 44 proposals. Three-word term frequency was not performed for the 44 proposals, reflecting the findings of the term frequency analysis of the 849 metadata entries.

Term Frequency Analysis

Among the one-word terms, “firm” was found the most often (2,621 occurrences), followed by “establishment” (1,498), “trade” (1,228), “economic” (1,224), “productivity” (935), “shock” (898), “plant” (846), “industry” (820), and “manufacturing” (641). These terms are mostly related to economy and business aspects. The term “firm” was also noted as the most frequent in the analysis of the 849 metadata entries. It is worth noting that “freight,” mostly found in records containing “commodity flow survey” in TRID searches, is not found in the top 25 terms. This means “freight” would be rarely used in economic or business-focused research, whereas it is often found in transportation-focused research.

Among two-word terms, “supply chain” was found most frequently (200 occurrences), followed by “commodity flow” (120), “economic review” (106), and “capacity utilization” (71), as shown in Figure 8. It is interesting that “labor market,” found the most frequently in the corresponding analysis of the metadata (Figure 3), is not found in the 44 proposals (Figure 7).

OPEN IN VIEWER

Figure 8.

Bigrams of two-word terms in the 44 proposals requesting confidential Commodity Flow Survey data (2011–2022).

Based on the results of the term frequency analysis and the TRID search, we can infer that the term “supply chain” is most likely used in research addressing economic and business issues, while “freight” is more commonly found in research focused on transportation issues. This distinction highlights the different focal points of studies using CFS data, whether public or confidential.

From a geographic perspective, supply chains involve not only the origin and destination of a shipment but also intermediate locations where shipments are temporarily stored or transferred between modes of transport. Capturing such information is challenging under the current CFS survey design. However, advances in technology (e.g., shipment tracking systems) and/or enhancements to the CFS survey design could make it feasible to collect these data—perhaps not for all shipments, but for a sufficiently large subset to enable meaningful analysis. Such improvements would also respond to the desire for more detailed geographic information identified during the human review of the 44 proposals.

Topic Modeling

For LDA modeling, the number of topics needed to be selected in advance. Models with topic counts ranging from 2 to 15 were examined. Common terms found in the topics across the models included “firm,”“industry,”“establishment,” and “business.” Among the topic models generated, Figure 9 shows the results of a six-topic LDA model, displaying only the top 10 terms for each topic based on beta probability values. The horizontal axis (beta) represents the probability of a term being included in the corresponding topic. All the six topics have “firm” in their top two terms based on the beta probability. Topic 5 includes “firm,”“plant,”“industry,”“employment,” and “economic,” and records the highest coherence score, −1.09, implying it would be the best at describing the underlying theme of the 44 proposals among the six topics.

OPEN IN VIEWER

Figure 9.

Six main topics from the latent Dirichlet allocation model applied to 44 proposals requesting confidential Commodity Flow Survey data (2011–2022).

Based on the topic models developed, thematic patterns shared across the 44 proposals were identified. The term “firm” consistently ranked highest across all topics, although its beta values varied in magnitude. Nearly all topics centered on economic or business themes, aligning with the qualitative review findings on research areas (see Table 1 for the qualitative categorization of research topics). While the six topics shared a common economic or business focus, distinctions emerged through unique terms within each topic. For example, Topic 1 was characterized by “innovation” and “production” and Topic 3 by “technology” and “location.” Drawing on the term composition, the topics can be labeled as follows: Topic 1—Firm-level production and innovation shock; Topic 2—Labor and supply in manufacturing markets; Topic 3—Technology and location in industrial productivity; Topic 4—Local business growth and economic shock; Topic 5—Business and employment in industrial plants; and Topic 6—Distribution and growth in economic shock.

A topic model is a powerful tool for document review, enhanced by computing power. However, it is important to note that analysts should possess a certain level of understanding about the content of the documents being analyzed. Without this knowledge, there is a risk of identifying non-existent themes or overlooking existing ones. Additionally, a topic model such as LDA relies on the bag-of-words approach, which disregards the order of words. Other NLP models, for example transformer-based models such as bidirectional encoder representations from transformers, process text in parallel and capture contextual information in a bidirectional or autoregressive manner and provide more comprehensive summaries for improving understanding of the content ( 26 ). Future research may benefit from employing methodologies that explicitly capture word order sequences, such as deep neural networks, to derive a more nuanced understanding of the semantic content of the text (26, 27).