There are no HTML tags present in this sample.<\/p>\n
Step 2:<\/strong> Remove punctuations.<\/em><\/p>\n Percentage and dollar symbols are substituted with word annotations to retain their importance in the textual data. Further, periods, semi-colons, and commas are removed. Regex is commonly applied to remove or replace punctuations.<\/p>\n Step3:<\/strong> Remove numbers. <\/em><\/p>\n Numbers present in the text should be removed as they do not have significant use for sentiment analysis. It is crucial to note that sentiment analysis seeks to understand the context in which the numbers are used. Additionally, before removing numbers, abbreviations representing orders of magnitude, such as million (commonly represented by \u201cm,\u201d \u201cmln,\u201d or \u201cmn\u201d), billion, or trillion, should be replaced with the complete word.<\/p>\n Step 4:<\/strong> Remove white spaces<\/em><\/p>\n Extra spaces such as tabs, line breaks, and new lines should be identified and removed to keep the text intact and clean. The stripWhitespace function in R can be utilized to can be used to eliminate unnecessary white spaces from the text<\/p>\n The cleansed data is free of punctuations and numbers, with useful substitutions, as shown in the table below:<\/p>\n $$\\small{\\begin{array}{l|l} \\textbf{Sentence}&\\textbf{Sentiment}\\\\ \\hline\\text{Cramo slipped to a pretax loss of a EUR million from a pretax profit of EUR million}&\\text{Negative}\\\\ \\hline\\text{Profit before taxes decreased to EUR million from EUR million the year before}&\\text{Negative}\\\\ \\hline\\text{Also, construction expenses have gone up in Russia}&\\text{Negative}\\\\ \\hline{\\text{Finnish Bore that is owned by the Rettig family has grown recently}\\\\ \\text{through the acquisition of smaller shipping companies}}&\\text{Positive}\\\\ \\hline{\\text{The plan is estimated to generate some EUR million USD million in cost}\\\\ \\text{savings on an annual basis}}&\\text{Positive}\\\\ \\hline\\text{Earnings per share EPS are seen at EUR}&\\text{Positive}\\\\\u00a0 \\end{array}}$$<\/p>\n After textual data is cleansed, it should be normalized. The normalization process in text processing entails the following:<\/p>\n Step 1:<\/strong> Lowercasing<\/em> the alphabet aids the computer to process identical words appropriately. For example, \u201cAND,\u201d \u201cAnd,\u201d and \u201cand\u201d).<\/p>\n Step 2:<\/strong> Stop words<\/em> such as \u201cthe,\u201d \u201cfor,\u201d and \u201care,\u201d usually are removed to reduce the number of tokens involved in the training set for ML training purposes. However, some stop words such as not<\/em>, more<\/em>, very<\/em>, and few<\/em> are not eliminated as they carry significant meaning in the financial texts that are useful for sentiment prediction.<\/p>\n Step 3:<\/strong> Stemming<\/em> a process of linguistic normalization, which reduces words to their word root word. For example, connection, connected, connecting word reduce to a common word “connect.”<\/p>\n Step 4:<\/strong> Lemmatization<\/em> is identical to stemming except that it removes endings only if the base form is present in a dictionary. Lemmatization is much more costly and advanced relative to stemming.<\/p>\n Step 5:<\/strong> Tokenization <\/em>is the process of breaking down a text paragraph into smaller chunks, such as words. A token is a single entity that is a building block for a sentence or a paragraph.<\/p>\n Exploratory data analysis (EDA) performed on text data provides insights on word distribution in the text. These word counts can be used to examine words that are most commonly and least commonly present in the texts, i.e., outliers.<\/p>\n The graph below is a frequency distribution of the sample tokens before stop words are eliminated.<\/p>\n A word cloud is a data visualization approach used for representing text data. Word clouds can be made to visualize the most informative words and their term frequency (TF) values. Varying font sizes can show the most commonly occurring words. Further, color is used to add more dimensions, such as frequency and length of words.<\/p>\n The feature selection process will eliminate common words and highlight useful words for better model training.<\/p>\n Which of the following visualizations is most likely<\/em> to be appropriate in the exploratory data analysis step if our objective is to create a visualization that shows the most informative words in the dataset based on their term frequency (TF) values?<\/p>\n \u00a0 \u00a0 A. Scatter plot.<\/p>\n \u00a0 \u00a0 B. Word cloud.<\/p>\n \u00a0 \u00a0 C. Document term matrix.<\/p>\n The correct answer is B.<\/strong><\/p>\n A word cloud is a common visualization when working with text data as it can be made to visualize the most informative words and their TF values. Varying font sizes can show the most commonly occurring words in the dataset, and the color is used to add more dimensions, such as frequency and length of words.<\/p>\n A is incorrect.<\/strong>\u00a0 A scatter plot is a two-dimensional chart that can be employed to summarize and approximately measure relationships between two or more features.<\/p>\n C is incorrect.<\/strong>\u00a0A document term matrix (DTM) is a matrix where each row belongs to a text file, and each column represents a token. The number of rows is equivalent to the number of text files in a sample text dataset. The number of columns is equal to the number of tokens from the BOW built using all the text files in the same token is present in each document. A DTM is not a visualization tool.<\/p>\n<\/blockquote>\n Reading 7: Big Data Projects<\/p>\n LOS 7 (e) Describe preparing, wrangling, and exploring text-based data for financial forecasting<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" [vsw id=”ifHmwpgHWYY” source=”youtube” width=”611″ height=”344″ autoplay=”no”] Sentiment analysis refers to the analysis of opinions or emotions from text data. In other words, it refers to how positive, negative, or neutral a particular phrase or statement is regarding a \u201ctarget.\u201d Such…<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[102,229],"tags":[216,270,268,230,269],"class_list":["post-12306","post","type-post","status-publish","format-standard","hentry","category-cfa-level-2","category-quantitative-method","tag-cfa-level-2","tag-exploring-textual-data-for-financial-forecasting","tag-preparing-textual-data-for-financial-forecasting","tag-quantitative-method","tag-wrangling-textual-data-for-financial-forecasting","blog-post","no-post-thumbnail","animate"],"acf":[],"yoast_head":"\nText Wrangling<\/h2>\n
Data Exploration<\/h2>\n
Exploratory Data Analysis<\/h3>\n
![\"frequency](\"https:\/\/analystprep.com\/study-notes\/wp-content\/uploads\/2021\/03\/Img_110-1.jpg\")
From the figure, we observe that \u201ccurrencysign,\u201d and \u201cmillion\u201d are the most repeated words due to the financial nature of the data.<\/p>\n\n
Question<\/h2>\n
Solution<\/h3>\n