Upper Case, Lower Case and Proper Case

In this post I will show you how to use VBA functions "UCase", "LCase" and worksheet function Proper to format the text in excel sheet. I will explain this using below example. Assume we have list of names like this.

So we have 10 names in sheet 1. As you can see that names are not in a proper format. So now let's look at how to convert this 10 names to upper case first. You can use below code to do that.

Sub ConvertToUpperCase() Dim WS_Input As Worksheet Dim WS_Result As Worksheet Dim i As Integer Set WS_Input = Worksheets("Sheet1") Set WS_Result = Worksheets.Add For i = 1 To 10     WS_Result.Range("A" & i).Value = UCase(WS_Input.Range("A" & i).Value) Next i End Sub

So this will convert the names to upper case and list them in a new sheet. So this is the result you will get after running the code.

You can see that all the letters have converted to upper case. So now let's look at how to convert that names to lower case.  Below code will do that.

Sub ConvertToLowerCase() Dim WS_Input As Worksheet Dim WS_Result As Worksheet Dim i As Integer Set WS_Input = Worksheets("Sheet1") Set WS_Result = Worksheets.Add For i = 1 To 10     WS_Result.Range("A" & i).Value = LCase(WS_Input.Range("A" & i).Value) Next i End Sub

You can change the limits of for next loop to do this for any amount of rows. Also you can modify the code to replace existing text in sheet 1 with result text. After running this macro result will look like this.

So now let's look at how to convert these names to proper case or title case. This is little different than other two. Because in earlier two cases, we used VBA functions. But this time we need to use a worksheet function. Because of that Proper(WS_Input.Range("A" & i).Value) will not work this time. Instead we have to use Application.WorksheetFunction.Proper(WS_Input.Range("A" & i).Value). Here is the full code.

Sub ConvertToProperCase() Dim WS_Input As Worksheet Dim WS_Result As Worksheet Dim i As Integer Set WS_Input = Worksheets("Sheet1") Set WS_Result = Worksheets.Add For i = 1 To 10     WS_Result.Range("A" & i).Value = Application.WorksheetFunction.Proper(WS_Input.Range("A" & i).Value) Next i End Sub

You will get below result after running this macro.

So modify above codes to suit with your requirements and improve your workbooks.

Web Scraping - Collect Options Inside Drop-down Lists

When we develop web scraping scripts we may need to collect items (options)  inside drop-down lists. So let's look at how to develop a code to collect options inside a select tag.

If you are new to Web Scraping, please read this post first.

Web Scraping - Basics

This is how drop-downs look like.

Below is the source code of above drop-down list.

<select d-start="1"  name="qty" class="name1"><option value="1" selected="selected">Quantity: 1</option><option value="2">Quantity: 2</option><option value="3">Quantity: 3</option></select>

So if you need to collect options inside this drop down list, you can use below code.

Dim QuantityString As String Set objIE = CreateObject("InternetExplorer.Application") objIE.Top = 0 objIE.Left = 0 objIE.Width = 800 objIE.Height = 600 objIE.Visible = True objIE.Navigate ("Your url here")   Do DoEvents Loop Until objIE.readystate = 4 QuantityString = objIE.document.getElementsByClassName("name1")(0).innerText

So what it does is, it will store the options inside the drop-down to variable call "QuantityString"

If you want to see the result in the message box use below line.

MsgBox QuantityString

So it will look like this.

However sometimes these drop-down lists may in disabled status. So now let's look at how to detect if a particular drop down list is in a disable status. Here below is a sample HTML code of a disabled drop-down list.

<select disabled="disabled" dstart="1" class="name2"><option value="1" selected="selected">Quantity: 1</option><option value="2">Quantity: 2</option><option value="3">Quantity: 3</option></select>

So here what we can do is, first assign whole code inside Select tag to a new variable. Then check that string for disabled="disabled". You can use inbuilt VBA function like InStr to check that.

This is how you can do it.

Dim DisableCheckingString As String Set objIE = CreateObject("InternetExplorer.Application") objIE.Top = 0 objIE.Left = 0 objIE.Width = 800 objIE.Height = 600 objIE.Visible = True objIE.Navigate ("Your url here")   Do DoEvents Loop Until objIE.readystate = 4 DisableCheckingString = objIE.document.getElementsByClassName("name2")(0).outerhtml If InStr(1, DisableCheckingString, "disabled=""disabled", vbTextCompare) > 0 Then     msgbox "This combo box is disabled" End If

You can alter this code to suit with your requirements. Because you may not need to show message box when the drop down list is disabled in real life examples. Instead you may need to skip those drop downs or may be you will need to mention about them in the result pages you create.

Web Scraping - Collecting Data From a Webpage

Web pages have various HTML tags. So the data we want to collect can be contained in any of these tags. Because of this you will need to carefully examine the HTML structure of the webpages before develop codes.

Web browsers have special tool which helps us to look at the structure of the web pages very easily. For an example, think you need to collect names of the people which is shown in a webpages. In this example each name is shown in separate page. But structure of each page is identical. So what you need to do first is, take your cursor near the name. Then write click. You will see this kind of options.

Then click on the Inspect.

You will able to see the structure of the web-page. Assume the code relate to name is look like this.

<span itemprop="full name" class="full name block">James Smith</span>

So I will show you how to collect name from this kind of structure. Here we can extract name using class name as follows.

objIE.document.getElementsByClassName("name block")(0).innerText

objIE should be defined before like

Set objIE = CreateObject("InternetExplorer.Application")

If you are not familiar with those basic web scraping, you can learn them from my earlier post.

Web Scraping - Basics

So let's now look at how to extract data from hyperlinks. Assume telephone numbers are stored in a hyperlink like below.

<a class="click-link" data-gaaction="ourbusiness" data-gaevent="pho_number" href="tel:9128602884" itemprop="phone">912-860-2884</a>

We can use below type of VBA code to get the telephone number.

Dim telephone_number As String Set Alllinks = objIE.document.getElementsByTagName("A") For Each Hyperlink In Alllinks     If InStr(1, Hyperlink.href, "tel:", vbTextCompare) > 0 Then         telephone_number = Hyperlink.innerText         Exit For     End If Next

Also we can use following technique to get text inside div tags.

Dim AllString As String         Set AllDiv = objIE.document.getElementsByTagName("div") For Each DivTag In AllDiv     AllString = DivTag.innerText Next

Then we can use functions like InStr, Split etc.. to extract relavent information from that "AllString" variable.

Web Scraping Techniques

In this post we are going to discuss about few more techniques used in web scraping. If you are new to web scraping, please read my earlier posts from below.

Web Scraping - Basics

Useful References for VBA Web Scraping

In this post I'm going to explain about more advance techniques.

When we do the data mining, sometimes we need to go to certain websites, put some value in text box and click on "Search" button. Then website gives list of results. These results sometimes may in several pages. If you want to automate this kind of process, you can develop a code to go to that url, then put value to text box, and then write a code to click button and so and so. You can learn how to write this kind of code from my earlier post.

But sometimes this can be easier than you think. Because it may be possible to get result of each page by changing the url. For an example if the website gives results in several pages, url's of each page sometimes have part like "?page=1", "?page=2" etc. So you can develop a program to get data from each page directly. And if you are searching for some value in textbox, then that value also can contained in the url. So it is always better to carefully look at the url and try to find some patterns.

Then sometimes we need to open the each result in separate pages to get all the information of each and every result. Think website have hyperlink like "View" for each result. So we can collect url of each of this "View" link from following code.

Set objIE = CreateObject("InternetExplorer.Application") url="url of particular result page" objIE.Navigate (url) Dim WS as worksheet set WS=activesheet 'wait to load page... Do DoEvents Loop Until objIE.readystate = 4 Set Alllinks = objIE.document.getElementsByTagName("A")     For Each Hyperlink In Alllinks         If InStr(Hyperlink.innerText, "View") > 0 Then             WS.Cells(row, 1).Value = Hyperlink.href             row = row + 1         End If     Next

So what it does is, it lists url of each result in a worksheet. Also you should have some idea about how many result pages the website will give. Then you can set suitable upper limit to the number of pages and should use some technique to exit the loop when there are no more result pages. Here below is example of such program.

Res = InputBox("Please enter number You want to search?", "") Set objIE = CreateObject("InternetExplorer.Application") objIE.Top = 0 objIE.Left = 0 objIE.Width = 1100 objIE.Height = 700 objIE.Visible = True Dim i, row As Integer Dim url, isdata As String Dim WS As Worksheet Set WS = Worksheets("DataBase") i = 1 row = 1 For i = 1 To 100     'genarate url...     url = "your website url+ ?page=" + CStr(i) + "There may be some additional parts of url here" + CStr(Res)     objIE.Navigate (url)         'wait to load page...     Do     DoEvents     Loop Until objIE.readystate = 4         'collect links...     isdata = "n"         Set Alllinks = objIE.document.getElementsByTagName("A")     For Each Hyperlink In Alllinks         If InStr(Hyperlink.innerText, "View") > 0 Then             WS.Cells(row, 1).Value = Hyperlink.href             isdata = "y"             row = row + 1         End If     Next     If (isdata = "n") Then         i = 101     End If Next i objIE.Quit

Determine whether characters inside a cell are bold or regular

Sometimes we need to determine whether characters inside a cell are bold or regular. And some cells can contain both bold and regular characters. Here is an example.

So today we are going to develop a macro to determine whether each and every character inside a cell is regular or bold. What it does is, it outputs each letter in immediate window and tells whether that each letter is bold or regular. Here below is the full code.

Dim i As Integer Dim Sentence_Length As Integer Sentence_Length = Len(Range("B2").Value) For i = 1 To Sentence_Length     Debug.Print Range("B2").Characters(i, 1).Text         If Range("B2").Characters(i, 1).Font.Bold = True Then         Debug.Print "Bold"     Else         Debug.Print "Regular"     End If Next i

Below is the explanation of above code.

First we need to define our variables. Then below line of code measure the number of characters inside B2 cell.

Sentence_Length = Len(Range("B2").Value)

And this for next loop, loop through all the letters in B2 cell.

For i = 1 To Sentence_Length     Next i

Below line prints the each character in immediate window. Please note that this program will consider spaces also as characters.

Debug.Print Range("B2").Characters(i, 1).Text

This part of the code checks whether the character is bold or regular and print the result in immediate window.

If Range("B2").Characters(i, 1).Font.Bold = True Then         Debug.Print "Bold" Else         Debug.Print "Regular" End If

Here is an image of immediate window after running the macro.

So this is just a very simple example. You can alter this code to suit with your requirements.

Useful References for VBA Web Scraping

        This post is to give small tip related to web scraping. If you are  new to web scraping please see my earlier post which explain everything from beginning. Below is the link to that post.

Web Scraping - Basics

In this post I will explain you what are the best VBA references you should add when you develop a Visual Basic Application to scrape data from websites. Adding these references are not mandatory. But it will make your life easier. So here are the list of useful references.

• Visula Basic for Applications
• Microsoft Excel 15.0 Object Library
• OLE Automation
• Microsoft Office 15.0 Object Library
• Microsoft Forms 2.0 Object Library
• ietag 1.0 Type Library
• iextag 1.0 Type Library
• Microsoft HTML Object Library
• Microsoft Internet Controls

As you can see that there are numbers like 15.0 etc. These numbers can vary according the versions you have installed in your computer.

Also when you open your VBA editor you will notice that first four items of the list are automatically ticked by default. So you don't need to add them manually.

And when you insert form for the first time to your project, VBA editor will automatically add reference to Microsoft Forms 2.0 Object Library. So you also don't need to worry about adding the reference to that library.

Ultimately you have shorter list to add references manually. Here is that list

• ietag 1.0 Type Library
• iextag 1.0 Type Library
• Microsoft HTML Object Library
• Microsoft Internet Controls

Once you add references, your references window should look like this.

Shapes

We often need to use shapes in Excel worksheets. You can insert various kinds of shapes to Excel worksheets from insert menu. Also you can use your own pictures as shapes. Here are few VBA examples related to shapes.

If you want to assign macro programmatically to your shape you can use below method.

Sub AssignMacro() ActiveSheet.Shapes("Picture 20").OnAction = "'" & ActiveWorkbook.Name & "'!Macro1" End Sub

You may want to know the row number of cell of top left of your shape. You can use below code for that.

Sub TopLeftCellRowNumber() MsgBox ActiveSheet.Shapes("Picture 20").TopLeftCell.Row End Sub

Below code will select all your shapes which have "picture" in its name.

It will select one shape at a time. Then will wait for 1 second and will select next shape.

Sub SelectPictureShapes() Dim sh As Shape     For Each sh In ActiveSheet.Shapes         If InStr(1, sh.Name, "picture", vbTextCompare) > 0 Then             sh.Select             Application.Wait Now + TimeValue("00:00:01")         End If Next End Sub

Sometimes you may need to know the name of the clicked shape in your program. Because sometimes your program will need to do different things depend on what shape is clicked. So below code will show the name of clicked shape in a message box. You can develop this to suit with your needs.

msgbox ActiveSheet.Shapes(Application.Caller).Name