Golang Parsing Strings

I have been working with Golang strings and how to manipulate them. Working from other blogs posts I’ve found. When I sit down to code, I seem to forget everything from the previous session. This post is to be a cheat sheet for myself on string manipulation.

References Can be found here

  • https://medium.com/@TobiasSchmidt89/effective-text-parsing-in-golang-163d13784288
  • https://golangcode.com/writing-to-file/
  • https://medium.com/golangspec/in-depth-introduction-to-bufio-scanner-in-golang-55483bb689b4

FILE with data being used

To give a practical application to this post, I have included a rudimentary SSH auth log I found from the internet (googles). I have repeated lines and changed data so it could be manipulated more.

Find it below SSH auth Log Code to play with this auth log based on the below can be found here on
Github

Ingesting a file

So first off we need data to play with.
We could work with a simple string like:

testString := "a long string"

But that is no fun.

Opening a File

So this is how to open a file & check for errors:

1
2
3
4
file, err := os.Open("sshinvalid.txt")
if err != nil {
	fmt.Println("opening file error", err)
}

Reading a file

If the file is open, we will now want to read the file.

Use the scanner package:

Package scanner provides a scanner and tokenizer for UTF-8-encoded text. It takes an io.Reader providing the source, which then can be tokenized through repeated calls to the Scan function. For compatibility with existing tools, the NUL character is not allowed.

In short: it scans UTF-8-encoded text

Need to read the file and split it (file is from the above example)

1
2
scanner := bufio.NewScanner(file)
scanner.Split(bufio.ScanLines)

Reading line by line + Adding to an array

Generally the data needs to be stored into a variable so that it can be manipulated. Create a string slice to place the data in, then loop through each line.

scanner.Scan() holds the Boolean of true or false if there is text scanner.Text() holds the data we want to manipulate

Create a string slice to place the data in, then loop through each line of Scanner.Text()

1
2
3
4
5
var txtlines []string

for scanner.Scan() {
	txtlines = append(txtlines, scanner.Text())
}

Closing file

Once placing the data into a variable, we can close the file.
I prefer to store data into a variable because it is easier for me to read. You are welcome to edit the data within the for scann.Scan() loop directly instead of storing it into a variable.

1
file.Close

Code up to this point:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
func main() {

	file, err := os.Open("sshinvalid.txt")
	if err != nil {
		fmt.Println("opening file error", err)
	}

	scanner := bufio.NewScanner(file)
	scanner.Split(bufio.ScanLines)
	var txtlines []string

	for scanner.Scan() {
		txtlines = append(txtlines, scanner.Text())
	}
	file.Close()

What has been done so far

  • Opening a file
  • reading the data in line by line
  • storing the lines into an variable (slice)
  • closing the file

Manipulating Data

Finding a value in string

To search through a string for a value, the strings.Contains() function is used.

scanner.Text(), "word" is how the syntax would be for using the Scan.

Since we have data in a slice, we will have to loop through it.

Example slice, which we loop through to test if string is in a string.

1
2
3
4
	testSlice:= []string{"the bbq", "the string has", "not much at all"}
	for _, value := range testSlice{
	fmt.Println(strings.Contains(value,"string"))
	}

The Go Playground code to see live:

  • https://play.golang.org/p/D2nzknlclvH

To print the line, an IF ELSE statement can be used:

1
2
3
if strings.Contains(value, "string") == true {
			fmt.Println(value)
		}

Separating via whitespace

Similar to linux: cut -d “ “

To separate a string via white space (words) use strings.Fields() function. Syntax being : strings.Fields(value)

Using the the first value from testSlice: testSlice[0]:

1
wordBroken := strings.Fields(testSlice[0])

Using testSlice from before, but looping through it

1
2
3
4
5
6
7
8
for _, value := range testSlice  {
		wordBreakDown := strings.Fields(value)
		fmt.Println("==breaking sentence into words==")
		for _, value := range wordBreakDown {
			fmt.Println(value)
		}

	}

The Go Playground code The output for this is below:

Separating by a delimiter

I definitely had to research this.

To separate by something other than a space, use the strings.FieldFunc(). Which comes in the format of func FieldsFunc(s string, f func(rune) bool) []string

The example given is from:

package main

import ( “fmt” “strings” “unicode” )

func main() { f := func(c rune) bool { return !unicode.IsLetter(c) && !unicode.IsNumber(c) } fmt.Printf(“Fields are: %q”, strings.FieldsFunc(“ foo1;bar2,baz3…”, f)) }

This is really difficult to understand.
It is easier to understand this function as giving it a a second parameter of a function that returns a bool value (true or false). Another way, is a function that confirms true or false if a separator is in the stings. (least that is how I simplify it.) This function is within the function.

So cutting a comma from testSlice:= "the bbq, the string has, not much at all" would look like:

1
2
3
4
5
6
7
8
9
	testSlice := "the bbq, the string has, not much at all"

	cuttingByTwo := strings.FieldsFunc(testSlice, func(r rune) bool {
		if r == ',' {
			return true
		}
		return false
	})
	fmt.Println(cuttingByTwo)

Which outputs:

The Go Playground code

Separate by 2 delimiters (or more)

The process is the same as one delimiter that is not space, you just add an OR statement with the 2nd delimiter.

The example string will be: testSlice:= "we want just the port number: [8080]"

1
2
3
4
5
6
7
8
9
testSlice:= "we want just the port number: [8080]"

	cuttingByTwo := strings.FieldsFunc(testSlice, func(r rune) bool {
		if r == '[' || r == ']' {
			return true
		}
		return false
	})
	fmt.Println(cuttingByTwo[1])

The Go Playground code

To do this via the command line you would do:

echo "we want just the port number: [8080]" | cut -d "[" -f2 | cut -d "]" -f1 8080

Hope this helps with parsing strings.

2020

Tcpdump Notes

I have been using tcpdump recently and wanted to note down some of the commands Y’know, for future reference.

Pivoting with SSH

Today I was trouble shooting a machine at work. I did not have access via RDP or VNC, so I used SSH to forward my traffic to the host so I could access a URL.

GitHub Actions

I participated in a DevSecOps type workshop on Saturday (May 9th) in which we created some GitHub Actions. This is a post to solidify the learning and be a c...

Threat Defense Workshop

On April 25th I was fortunate enough to participate in the Trend Micro Threat Defense workshop.

Incident Handling Certification

Since I blogged about my experience at OpenSoc, I wanted to expand on the value I found in my eLearnSecuirty Incident Response course. What you will find bel...

OpenSoc Experience

So Thursday (April 9th) I participated in an online blue team defense simulation event, known as OpenSOC.

Golang Parsing Strings

I have been working with Golang strings and how to manipulate them. Working from other blogs posts I’ve found. When I sit down to code, I seem to forget ever...

Welcome to Jekyll!

You’ll find this post in your _posts directory. Go ahead and edit it and re-build the site to see your changes. You can rebuild the site in many different wa...

Back to Top ↑