Want to see in Go caused by an innocent append of data to an array? Can this happen in Rust? Check this out! disappearing data I was playing with Go and Rust. When I was reading , the following question popped up in my head: slices intro in Go When is safe? For Go: append(...) After deeper thinking, I realized, that the answer is as for the next question! the same What's the difference between and ? For Rust: Vec<SomeType> &[SomeType] The text answers the questions giving an interesting insight into design differences between the two languages. The case study below shows caused by naive use of . data corruption append(...) Don't forget to look up the attached ! GIT repo Album software in Go Let's write simple album software to manage photos. This will be a web server in Go. Meet who is passionate about photography. Bob Bob likes to take . He gives them names, to be able to find them easily at drunken parties. photos DateTime time.Time
} // Photo was taken in particular time using camera. // Has non-unique name given by user. type Photo struct { ID int Name string Bob groups photos into , such as "Nature", "Cars", or "Egypt". albums Photos []Photo
} // Album is collection of photos sorted by time. // Has non-unique name given by user. type Album struct { ID int Name string He wants to easily find photos from his visit to Boston (which took place from 2019-06-15 to 2019-06-17). } // GetPhotos returns photos in given time interval. func (album *Album) GetPhotos (start, end time.Time) [] Photo { var i, j int // because album is already sorted by time, // we just calculate i and j containing photos // from the requested interval. ... // omitted return p[i:j] Bob wants to impress friends by showing vacation pictures with pyramids. photos := album.GetPhotos( ) } // GetHolidayPhotos gets photos from June. func GetHolidayPhotos (album *Album, year int ) [] Photo { time.Date(year, 6 , 1 , 0 , 0 , 0 , 0 , time.UTC), time.Date(year, 7 , 1 , 0 , 0 , 0 , 0 , time.UTC), return photos Seems OK. Just one more thing... Bob would like also to show photos from at his mother's home. Fortunately, we can adapt with : the winter holidays GetHolidayPhotos append(...) ) decemberPhotos := album.GetPhotos( time.Date(year, 12 , 1 , 0 , 0 , 0 , 0 , time.UTC), time.Date(year+ 1 , 1 , 1 , 0 , 0 , 0 , 0 , time.UTC), // photos already has photos from June. photos = append (photos, decemberPhotos...) Of course, we wrote nice like this: unit tests want := []Photo{
		{ }, { },
	} }
} func TestGetHolidayPhotos (t *testing.T) { a := createAlbum() // factory creating example album got := GetHolidayPhotos(a, 2021 ) ID: 2 , Name: "Camellia" , DateTime: time.Date( 2021 , 6 , 12 , 14 , 0 , 0 , 0 , time.UTC), ..., // omitted ID: 7 , Name: "More snow" , DateTime: time.Date( 2021 , 12 , 15 , 14 , 0 , 0 , 0 , time.UTC), if !reflect.DeepEqual(got, want) { t.Errorf( "Got = %v, want %v" , got, want) Tests , git commit, push, done! passing The next day... You had a nightmare where Bob's photos from Egypt were eaten by mummies. Maybe something is wrong with our code? is a , in other words, it shouldn't modify the album. Let's make additional unit test: GetHolidayPhotos query album := createAlbum() serializedAlbum, err := json.Marshal(album) } postmortemSerializedAlbum, err := json.Marshal(album) } t.Errorf( )
	}
} func TestGetHolidayPhotosNotMutatingInput (t *testing.T) { // (1) we serialize album to JSON. if err != nil { t.Fatalf( "Got error %v" , err) // (2) perform the query. GetHolidayPhotos(album, 2021 ) // (3) serialize after the query. if err != nil { t.Fatalf( "Got error %v" , err) // (4) serialized album shouldn't change. if !bytes.Equal(postmortemSerializedAlbum, serializedAlbum) { "Got postmortemSerializedAlbum = %v, want serializedAlbum = %v" , string (postmortemSerializedAlbum[:]), string (serializedAlbum[:]), What do expect - will it pass? you FAIL $ go test --- FAIL: TestGetHolidayPhotosNotMutatingInput ( 0.00 s) album_test.go: 155 : Got postmortemSerializedAlbum = { "ID" : 0 , ... exit status 1 It failed! is input data! GetHolidayPhotos corrupting What happened? Let's look at what happens inside in the failing test. GetHolidayPhotos First, we get photos from June: photos := album.GetPhotos( /* June date range */ ) Now points to contents of - is equal to . photos album.Photos photos album.Photos[2..4] It's how Go sees slice: photos The diagram has a few parts: All items ( , , ) are in . For example, has a photo of camellia. gray orange blue album.Photos album.Photos[2] items are referenced by the slice. For example, has a photo of daisy. The of the slice is . Orange photos photos[1] length len(photos) = 2 Next December photos are appended: photos = append (photos, decemberPhotos...) elements are treated by Go as free slots for . Blue append Let's see how slice look after the append: photos Some elements were replaced with photos from December. Thus indeed contains photos from June and December. blue photos However, as you can see from the diagram, this operation destroyed the contents of ! album.Photos Photo of was discarded, and there is a duplicate photo of snow from 2021-12-11 in and . "Forget Me Not" album.Photos[4] album.Photos[6] This effect was from perspective. Below I show a simple to avoid such issues from perspective. already described a technical rule of thumb a semantic What is array duality? I propose in this article two forms in which you can meet arrays: - you append to the array, owned can - you append to the array. borrowed cannot are used to store data, which might shrink or expand. It could be a repository of users or a list of currently opened files. Owned are pointing to another array or its subarray. It's a kind of view. Could be used to pass one sentence from a string to a function. Might be returned as a query result. Borrowed In our album software is array, because the album contains/manages photos. album.Photos owned On the other hand, gives array - it returns the slice . This way it does not have to allocate array - instead, it points to the original array from the album. album.GetPhotos(start_time, end_time) borrowed album.Photos[i:j] a new Let's look, at how the duality looks in Go and Rust. In Go In Go, we need a bit of self-discipline from a developer. There is no clear indicator, of whether an array is owned or borrowed. However, sometimes it's easy to infer this from context. In case of , we know that album contains photos. We could add a method to make this explicit. Album AddPhoto(photo Photo) When we subslice an array, usually the result is array, as in . borrowed album.GetPhotos The exception is subslicing used to pop elements from the end of an array: // myPhotos[5], myPhotos[6], myPhotos[7], ..., are discarded. myPhotos = myPhotos[: 5 ] In Rust This whole "array duality" was inspired by the ownership model in Rust. The distinction between and array is explicitly marked by type. owned borrowed In Rust array is represented by . You can push and pop elements: owned Vec<Photo> photos.push(Photo { ... }); let mut photos = Vec ::new(); array is represented by a slice . You get it by subslicing an array: A borrowed &[Photo] let june_photos = &photos[ 2 .. 5 ]; However, Rust forbids to append to array: borrowed june_photos.push(Photo { ... }); gives compile error: error[E0599]: no method named push found for reference &[ Photo ] in the current scope That's it! Let's summarize it with a table: Bugfix Let's make a walk-through and: ask whether a given array is or , owned borrowed make sure, that we do append to arrays. not borrowed This way the bug in the Go server should be fixed. The original code: photos := album.GetPhotos( ) decemberPhotos := album.GetPhotos( ) } // GetHolidayPhotos gets photos from June and December. func GetHolidayPhotos (album *Album, year int ) [] Photo { time.Date(year, 6 , 1 , 0 , 0 , 0 , 0 , time.UTC), time.Date(year, 7 , 1 , 0 , 0 , 0 , 0 , time.UTC), // (1) album.GetPhotos gives *borrowed* array. ... // omitted photos = append (photos, decemberPhotos...) // (2) we are appending to *borrowed* array - BAD! return photos We must ensure, that we do not to a borrowed array. Fixed code: append(...) junePhotos := album.GetPhotos( ) decemberPhotos := album.GetPhotos( ) } // GetHolidayPhotos gets photos from June and December. func GetHolidayPhotos (album *Album, year int ) [] Photo { var photos []Photo // (1) we create a new empty array. // It's *owned* array. time.Date(year, 6 , 1 , 0 , 0 , 0 , 0 , time.UTC), time.Date(year, 7 , 1 , 0 , 0 , 0 , 0 , time.UTC), photos = append (photos, junePhotos...) // (2) Appending to *owned* array - OK. time.Date(year, 12 , 1 , 0 , 0 , 0 , 0 , time.UTC), time.Date(year+ 1 , 1 , 1 , 0 , 0 , 0 , 0 , time.UTC), photos = append (photos, decemberPhotos...) // (3) Appending to *owned* array - OK. return photos Notice that appending to empty slice in Go is essentially a copy of the array in the argument. Now test TestGetHolidayPhotosNotMutatingInput passes! Rewrite It in Rust! Out of curiosity, I've ported the software to Rust. and structs are defined similarly as in Go. It's noteworthy that has explicitly an array of photos: Album Photo Album owned } /// Album is a collection of photos sorted by time. /// Has a non-unique name given by the user. #[derive(Debug, PartialEq, Eq, Clone, Serialize, Deserialize)] pub struct Album { pub id: u64 , pub name: String , pub photos: Vec <Photo>, returns array, notice returned type: album.get_photos(..) borrowed start: DateTime<Utc>,
        end: DateTime<Utc>,
    ) -> &[Photo] { }
} impl Album { /// Returns photos in given time interval. pub fn get_photos ( & self , let mut i: usize ; let mut j: usize ; // because album is already sorted by time, // we just calculate i and j containing photos // from the requested interval. ... // omitted & self .photos[i..j] And : get_holiday_photos DateTime::from_utc( Utc,
        ),
        DateTime::from_utc( Utc,
        ),
    );    
    photos.extend_from_slice(june_photos); );    
    photos.extend_from_slice(december_photos); photos
} /// Gets photos from June and December. pub fn get_holiday_photos (album: &Album, year: i32 ) -> Vec <Photo> { let mut photos = Vec ::new(); // (1) we create a new empty array. // It's *owned* array, because it's type is `Vec<Photo>`. let june_photos = album.get_photos( NaiveDate::from_ymd(year, 6 , 1 ).and_hms( 0 , 0 , 0 ), NaiveDate::from_ymd(year, 7 , 1 ).and_hms( 0 , 0 , 0 ), // (2) Appending to *owned* array - OK. let december_photos = album.get_photos( ... // omitted // (3) Appending to *owned* array - OK. Trade-off This case study highlights the differences between Go and Rust. Rust requires a developer to declare explicitly how data will be used. Thus the compiler can statically verify this promise - which is useful for a team of developers. Go requires more self-discipline (the most notable example is passing data among threads). Sometimes a library can take advantage of such declarations - for instance, JSON deserialization can take substring of input, instead of copying it (a.k.a. "zero-copy deserialization") - . a borrowed more in the documentation of library serde On the other hand, Go has a lower learning curve - no discussion about vectors, ownership, or async stuff. A lot of problems can be solved by , like array copying before append or . You do not have to know about the existence of in the standard library... defensive programming sharing memory by communicating cows Cows? Up to this point, I've discussed two disjoint scenarios: returning from function array, always owned returning from function array. always borrowed However, there is a third case: function might at runtime dynamically decide, whether to return or array. owned borrowed Bob would like to show his holiday photos, but only from summer. // GetHolidayPhotos gets photos from June and December. func GetHolidayPhotos ( album *Album, year int , june, december bool , ) [] Photo Instead of allocating always a new ( ) array, we can sometimes return array: owned borrowed ) } junePhotos := album.GetPhotos( ... // omitted // (1) `junePhotos` is borrowed slice from album.Photos. if june && !december { return junePhotos // (2) returning in this case *borrowed* array. // (3) Original code returning *owned* array. As was shown in the table, and arrays have the same type in Go. owned borrowed And in Rust? Which type should be returned? album: &Album, /// Gets photos from June and December. pub fn get_holiday_photos ( year: i32 , june: bool , december: bool , ) -> Vec <Photo> // or &[Photo] ? Let's try similar optimization as in Go: } if june && !december { return june_photos; // returning &[Photo] gives error error[E0308]: mismatched types because is a different thing than . Vec<Photo> &[Photo] This shows the in Rust caused by the requirement of declaration of an array type ( vs ). To know how to solve this, you must be aware that exist and read the mysteriously named article ! steeper learning curve an explicit owned borrowed cows "The Secret Life of Cows" Rust has a distinct type for such dynamic ownership: , which might contain or array. The first time I've met a cow in the wild was the search and replace string function in . Cow<[Photo]> owned borrowed library regex For the sake of completeness, I give there updated table: Conclusions The text described "array duality" in Go and Rust, showing when it is correct to append a new value to some array. The way the concept is handled in these post-2010 languages shows philosophical differences between them. Rust requires a developer to explicitly state the duality of each array in code. On the other hand, Go doesn't burden developers with such annotations, instead requires self-discipline from a developer. This article was first published here

ORANGE

Read My Stories

Nominated for 2022 - HackerNoon Contributor of the Year - Golang

Too Long; Didn't Read

Make resilience your competitive advantage

Go vs Rust: A Sto-array of Arrays

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

3 JavaScript Performance Mistakes You Should Stop Doing

A Simple Introduction to Arrays In JavaScript

Array Filter vs Array Reduce vs For Loop

Arrays in Ruby: Benefits and Costs to Use Them

Arrays vs. Linked Lists: Which Is Better?

3 JavaScript Performance Mistakes You Should Stop Doing

A Simple Introduction to Arrays In JavaScript

Array Filter vs Array Reduce vs For Loop

Arrays in Ruby: Benefits and Costs to Use Them

Arrays vs. Linked Lists: Which Is Better?

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps