skills/gosom__google-maps-scraper
1
0
Fork 0
mirror of https://github.com/gosom/google-maps-scraper.git synced 2026-09-19 07:27:12 +08:00
Code Issues Packages Projects Releases Wiki Activity

extracts more datapoints & adds csv export support

Browse Source
This commit is contained in:
Giorgos Komninos
2023-09-08 20:50:47 +03:00
parent 194cdf34a1
commit f09017d459
9 changed files with 427 additions and 66 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+50 -26
View File
@@ -2,23 +2,55 @@
![build](https://github.com/gosom/google-maps-scraper/actions/workflows/build.yml/badge.svg)
[![Go Report Card](https://goreportcard.com/badge/github.com/gosom/google-maps-scraper)](https://goreportcard.com/report/github.com/gosom/google-maps-scraper)
A command line google maps parser build using
A command line google maps scraper build using
[scrapemate](https://github.com/gosom/scrapemate) web crawling framework.
You can use this repository either as is, or you can use it's code as a base and
customize it to your needs
## **Maintainers wanted**
## Features
Google frequentyl changes the layout of the pages and the CSS selectors needs to be adjusted and I would like some help.
- Extracts many data points from google maps
- Exports the data to CSV, JSON or PostgreSQL
- Perfomance about 55 urls per minute (-depth 1 -c 8)
- Extendable to write your own exporter
- Dockerized for easy run in multiple platforms
- Scalable in multiple machines
Please report if the tool is broken or even better make a Pull Request with the fix.
A small request please. If you use or like the program please ⭐ the repository, it may help to find some maintainers.
Thanks
## Extracted Data Points
```
link
title
category
address
open_hours
website
phone
plus_code
review_count
review_rating
reviews_per_rating
latitude
longitude
cid
status
descriptions
reviews_link
thumbnail
timezone
price_range
data_id
images
reservations
order_online
menu
owner
complete_address
about
user_reviews
```
## Quickstart
@@ -54,7 +86,7 @@ try `./google-maps-scraper -h` to see the command line options available:
```
-c int
sets the concurrency. By default it is set to half of the number of CPUs (default 8)
sets the concurrency. By default it is set to half of the number of CPUs (default NUM_CPU)
-cache string
sets the cache directory (no effect at the moment) (default "cache")
-debug
@@ -64,9 +96,11 @@ try `./google-maps-scraper -h` to see the command line options available:
-dsn string
Use this if you want to use a database provider
-exit-on-inactivity duration
program exits after this duration of inactivity
program exits after this duration of inactivity(example value '5m')
-input string
is the path to the file where the queries are stored (one query per line). By default it reads from stdin (default "stdin")
-json
Use this to produce a json file instead of csv (not avalaible when using db)
-lang string
is the languate code to use for google (the hl urlparam).Default is en . For example use de for German or el for Greek (default "en")
-produce
@@ -75,19 +109,6 @@ try `./google-maps-scraper -h` to see the command line options available:
is the path to the file where the results will be written (default "stdout")
```
## Extracted Data
- Title: the title of the business
- Category: the category of the business
- Address: the address of the business
- OpenHours: the opening hours of the business
- WebSite: the website of the business
- Phone: the phone number of the business
- PlusCode: the plus code of the business
- ReviewCount: the number of reviews for the business
- ReviewRating: the rating of the business
- Latitude: the latitude of the business
- Longtitude: the longitude of the business
## Using Database Provider (postgreSQL)
@@ -164,11 +185,13 @@ Use an appropriate kubernetes cluster
## Perfomance
Expected speed with concurrency of 8 and depth 1 is 45 jobs/per minute.
Expected speed with concurrency of 8 and depth 1 is 55 jobs/per minute.
Each search is 1 job + the number or results it contains.
Based on the above:
if we have 1000 keywords to search with each contains 10 results => 1000 * 10 = 10000 jobs.
We expect this to take about 10000/45 ~ 222 minutes ~ 4 hours
if we have 1000 keywords to search with each contains 16 results => 1000 * 10 = 16000 jobs.
We expect this to take about 10000/55 ~ 291 minutes ~ 5 hours
If you want to scrape many keywords then it's better to use the Database Provider in
combination with Kubernetes for convenience and start multipe scrapers in more than 1 machines.
@@ -179,6 +202,7 @@ For more instruction you may also read the following links
- https://blog.gkomninos.com/how-to-extract-data-from-google-maps-using-golang
- https://blog.gkomninos.com/distributed-google-maps-scraping
- https://github.com/omkarcloud/google-maps-scraper/tree/master (also a nice project) [many thanks for the idea to extract the data by utilizing the JS objects]
## Licence
gmaps/entry.go
+241 -22
View File
@@ -4,30 +4,85 @@ import (
"encoding/json"
"fmt"
"runtime/debug"
"strings"
)
type Image struct {
Title string `json:"title"`
Image string `json:"image"`
}
type LinkSource struct {
Link string `json:"link"`
Source string `json:"source"`
}
type Owner struct {
ID string `json:"id"`
Name string `json:"name"`
Link string `json:"link"`
}
type Address struct {
Borough string `json:"borough"`
Street string `json:"street"`
City string `json:"city"`
PostalCode string `json:"postal_code"`
State string `json:"state"`
Country string `json:"country"`
}
type Option struct {
Name string `json:"name"`
Enabled bool `json:"enabled"`
}
type About struct {
ID string `json:"id"`
Name string `json:"name"`
Options []Option `json:"options"`
}
type Review struct {
Name string
ProfilePicture string
Rating int
Description string
Images []string
When string
}
type Entry struct {
Link string
Cid string
Title string
Categories []string
Category string
Address string
OpenHours map[string][]string
WebSite string
Phone string
PlusCode string
ReviewCount int
ReviewRating float64
Latitude float64
Longtitude float64
Status string
Description string
ReviewsLink string
Thumbnail string
Timezone string
PriceRange string
DataID string
Link string `json:"link"`
Cid string `json:"cid"`
Title string `json:"title"`
Categories []string `json:"categories"`
Category string `json:"category"`
Address string `json:"address"`
OpenHours map[string][]string `json:"open_hours"`
WebSite string `json:"web_site"`
Phone string `json:"phone"`
PlusCode string `json:"plus_code"`
ReviewCount int `json:"review_count"`
ReviewRating float64 `json:"review_rating"`
ReviewsPerRating map[int]int `json:"reviews_per_rating"`
Latitude float64 `json:"latitude"`
Longtitude float64 `json:"longtitude"`
Status string `json:"status"`
Description string `json:"description"`
ReviewsLink string `json:"reviews_link"`
Thumbnail string `json:"thumbnail"`
Timezone string `json:"timezone"`
PriceRange string `json:"price_range"`
DataID string `json:"data_id"`
Images []Image `json:"images"`
Reservations []LinkSource `json:"reservations"`
OrderOnline []LinkSource `json:"order_online"`
Menu LinkSource `json:"menu"`
Owner Owner `json:"owner"`
CompleteAddress Address `json:"complete_address"`
About []About `json:"about"`
UserReviews []Review `json:"user_reviews"`
}
func (e *Entry) Validate() error {
@@ -54,6 +109,7 @@ func (e *Entry) CsvHeaders() []string {
"plus_code",
"review_count",
"review_rating",
"reviews_per_rating",
"latitude",
"longitude",
"cid",
@@ -64,6 +120,14 @@ func (e *Entry) CsvHeaders() []string {
"timezone",
"price_range",
"data_id",
"images",
"reservations",
"order_online",
"menu",
"owner",
"complete_address",
"about",
"user_reviews",
}
}
@@ -79,6 +143,7 @@ func (e *Entry) CsvRow() []string {
e.PlusCode,
stringify(e.ReviewCount),
stringify(e.ReviewRating),
stringify(e.ReviewsPerRating),
stringify(e.Latitude),
stringify(e.Longtitude),
e.Cid,
@@ -89,6 +154,14 @@ func (e *Entry) CsvRow() []string {
e.Timezone,
e.PriceRange,
e.DataID,
stringify(e.Images),
stringify(e.Reservations),
stringify(e.OrderOnline),
stringify(e.Menu),
stringify(e.Owner),
stringify(e.CompleteAddress),
stringify(e.About),
stringify(e.UserReviews),
}
}
@@ -130,7 +203,9 @@ func EntryFromJSON(raw []byte) (entry Entry, err error) {
entry.Category = entry.Categories[0]
}
entry.Address = getNthElementAndCast[string](darray, 18)
entry.Address = strings.TrimSpace(
strings.TrimPrefix(getNthElementAndCast[string](darray, 18), entry.Title+","),
)
entry.OpenHours = getHours(darray)
entry.WebSite = getNthElementAndCast[string](darray, 7, 0)
entry.Phone = getNthElementAndCast[string](darray, 178, 0, 0)
@@ -148,9 +223,149 @@ func EntryFromJSON(raw []byte) (entry Entry, err error) {
entry.PriceRange = getNthElementAndCast[string](darray, 4, 2)
entry.DataID = getNthElementAndCast[string](darray, 10)
items := getLinkSource(getLinkSourceParams{
arr: getNthElementAndCast[[]any](darray, 171, 0),
link: []int{3, 0, 6, 0},
source: []int{2},
})
entry.Images = make([]Image, len(items))
for i := range items {
entry.Images[i] = Image{
Title: items[i].Source,
Image: items[i].Link,
}
}
entry.Reservations = getLinkSource(getLinkSourceParams{
arr: getNthElementAndCast[[]any](darray, 46),
link: []int{0},
source: []int{1},
})
orderOnlineI := getNthElementAndCast[[]any](darray, 75, 0, 1, 2)
if len(orderOnlineI) == 0 {
orderOnlineI = getNthElementAndCast[[]any](darray, 75, 0, 0, 2)
}
entry.OrderOnline = getLinkSource(getLinkSourceParams{
arr: orderOnlineI,
link: []int{1, 2, 0},
source: []int{0, 0},
})
entry.Menu = LinkSource{
Link: getNthElementAndCast[string](darray, 38, 0),
Source: getNthElementAndCast[string](darray, 38, 1),
}
entry.Owner = Owner{
ID: getNthElementAndCast[string](darray, 57, 2),
Name: getNthElementAndCast[string](darray, 57, 1),
}
if entry.Owner.ID != "" {
entry.Owner.Link = fmt.Sprintf("https://www.google.com/maps/contrib/%s", entry.Owner.ID)
}
entry.CompleteAddress = Address{
Borough: getNthElementAndCast[string](darray, 183, 1, 0),
Street: getNthElementAndCast[string](darray, 183, 1, 1),
City: getNthElementAndCast[string](darray, 183, 1, 3),
PostalCode: getNthElementAndCast[string](darray, 183, 1, 4),
State: getNthElementAndCast[string](darray, 183, 1, 5),
Country: getNthElementAndCast[string](darray, 183, 1, 6),
}
aboutI := getNthElementAndCast[[]any](darray, 100, 1)
for i := range aboutI {
el := getNthElementAndCast[[]any](aboutI, i)
about := About{
ID: getNthElementAndCast[string](el, 0),
Name: getNthElementAndCast[string](el, 1),
}
optsI := getNthElementAndCast[[]any](el, 2)
for j := range optsI {
opt := Option{
Enabled: getNthElementAndCast[int](optsI, j, 2, 1, 0, 0) == 1,
Name: getNthElementAndCast[string](optsI, j, 1),
}
if opt.Name != "" {
about.Options = append(about.Options, opt)
}
}
entry.About = append(entry.About, about)
}
entry.ReviewsPerRating = map[int]int{
1: int(getNthElementAndCast[float64](darray, 52, 3, 0)),
2: int(getNthElementAndCast[float64](darray, 52, 3, 1)),
3: int(getNthElementAndCast[float64](darray, 52, 3, 2)),
4: int(getNthElementAndCast[float64](darray, 52, 3, 3)),
5: int(getNthElementAndCast[float64](darray, 52, 3, 4)),
}
reviewsI := getNthElementAndCast[[]any](darray, 52, 0)
for i := range reviewsI {
el := getNthElementAndCast[[]any](reviewsI, i)
review := Review{
Name: getNthElementAndCast[string](el, 0, 1),
ProfilePicture: getNthElementAndCast[string](el, 0, 2),
When: getNthElementAndCast[string](el, 1),
Rating: int(getNthElementAndCast[float64](el, 4)),
Description: getNthElementAndCast[string](el, 3),
}
if review.Name == "" {
continue
}
optsI := getNthElementAndCast[[]any](el, 14)
for j := range optsI {
val := getNthElementAndCast[string](optsI, j, 6, 0)
if val != "" {
review.Images = append(review.Images, val)
}
}
entry.UserReviews = append(entry.UserReviews, review)
}
return entry, nil
}
type getLinkSourceParams struct {
arr []any
source []int
link []int
}
func getLinkSource(params getLinkSourceParams) []LinkSource {
var result []LinkSource
for i := range params.arr {
item := getNthElementAndCast[[]any](params.arr, i)
el := LinkSource{
Source: getNthElementAndCast[string](item, params.source...),
Link: getNthElementAndCast[string](item, params.link...),
}
if el.Link != "" && el.Source != "" {
result = append(result, el)
}
}
return result
}
//nolint:gomnd // it's ok, I need the indexes
func getHours(darray []any) map[string][]string {
items := getNthElementAndCast[[]any](darray, 34, 1)
@@ -202,6 +417,10 @@ func getNthElementAndCast[T any](arr []any, indexes ...int) T {
}
}
if len(indexes) == 0 || len(arr) == 0 {
return defaultVal
}
ans, ok := arr[indexes[0]].(T)
if !ok {
return defaultVal
gmaps/entry_test.go
+87 -1
View File
@@ -30,7 +30,7 @@ func Test_EntryFromJSON(t *testing.T) {
Title: "Kipriakon",
Category: "Restaurant",
Categories: []string{"Restaurant"},
Address: "Kipriakon, Old port, Limassol 3042",
Address: "Old port, Limassol 3042",
OpenHours: map[string][]string{
"Monday": {"12:30–10 pm"},
"Tuesday": {"12:30–10 pm"},
@@ -54,6 +54,78 @@ func Test_EntryFromJSON(t *testing.T) {
Timezone: "Asia/Nicosia",
PriceRange: "€€",
DataID: "0x14e732fd76f0d90d:0xe5415928d6702b47",
Images: []gmaps.Image{
{
Title: "All",
Image: "https://lh5.googleusercontent.com/p/AF1QipP4Y7A8nYL3KKXznSl69pXSq9p2IXCYUjVvOh0F=w298-h298-k-no",
},
{
Title: "Latest",
Image: "https://lh5.googleusercontent.com/p/AF1QipNgMqyaQs2MqH1oiGC44eDcvudurxQfNb2RuDsd=w224-h298-k-no",
},
{
Title: "Videos",
Image: "https://lh5.googleusercontent.com/p/AF1QipPZbq8v8K8RZfvL6gZ_4Dw6qwNJ_MUxxOOfBo7h=w224-h398-k-no",
},
{
Title: "Menu",
Image: "https://lh5.googleusercontent.com/p/AF1QipNhoFtPcaLCIhdN3GhlJ6sQIvdhaESnRG8nyeC8=w397-h298-k-no",
},
{
Title: "Food & drink",
Image: "https://lh5.googleusercontent.com/p/AF1QipMbu-iiWkE4DsXx3aI7nGaqyXJKbBYCrBXvzOnu=w298-h298-k-no",
},
{
Title: "Vibe",
Image: "https://lh5.googleusercontent.com/p/AF1QipOGg_vrD4bzkOre5Ly6CFXuO3YCOGfFxQ-EiEkW=w224-h398-k-no",
},
{
Title: "Fried green tomatoes",
Image: "https://lh5.googleusercontent.com/p/AF1QipOziHd2hqM1jnK9KfCGf1zVhcOrx8Bj7VdJXj0=w397-h298-k-no",
},
{
Title: "French fries",
Image: "https://lh5.googleusercontent.com/p/AF1QipNJyq7nAlKtsxxbNy4PHUZOhJ0k7HPP8tTAlwcV=w397-h298-k-no",
},
{
Title: "By owner",
Image: "https://lh5.googleusercontent.com/p/AF1QipNRE2R5k13zT-0WG4b6XOD_BES9-nMK04hlCMVV=w298-h298-k-no",
},
{
Title: "Street View & 360°",
Image: "https://lh5.googleusercontent.com/p/AF1QipMwkHP8GmDCSuwnWS7pYVQvtDWdsdk-CUwxtsXL=w224-h298-k-no-pi-23.425545-ya289.20517-ro-8.658787-fo100",
},
},
OrderOnline: []gmaps.LinkSource{
{
Link: "https://foody.com.cy/delivery/lemesos/to-kypriakon?utm_source=google&utm_medium=organic&utm_campaign=google_reserve_place_order_action",
Source: "foody.com.cy",
},
{
Link: "https://wolt.com/en/cyp/limassol/restaurant/kypriakon?utm_source=googlemapreserved&utm_campaign=kypriakon",
Source: "wolt.com",
},
},
Owner: gmaps.Owner{
ID: "102769814432182832009",
Name: "Kipriakon (Owner)",
Link: "https://www.google.com/maps/contrib/102769814432182832009",
},
CompleteAddress: gmaps.Address{
Borough: "",
Street: "Old port",
City: "Limassol",
PostalCode: "3042",
State: "",
Country: "CY",
},
ReviewsPerRating: map[int]int{
1: 37,
2: 16,
3: 27,
4: 60,
5: 256,
},
}
raw, err := os.ReadFile("../testdata/raw.json")
@@ -63,6 +135,20 @@ func Test_EntryFromJSON(t *testing.T) {
entry, err := gmaps.EntryFromJSON(raw)
require.NoError(t, err)
require.Len(t, entry.About, 10)
for _, about := range entry.About {
require.NotEmpty(t, about.ID)
require.NotEmpty(t, about.Name)
require.NotEmpty(t, about.Options)
}
entry.About = nil
require.Len(t, entry.UserReviews, 8)
entry.UserReviews = nil
require.Equal(t, expected, entry)
}
go.mod
+1 -2
View File
@@ -8,7 +8,6 @@ require (
github.com/gosom/scrapemate v0.5.1
github.com/jackc/pgx/v5 v5.3.1
github.com/playwright-community/playwright-go v0.2000.1
github.com/shopspring/decimal v1.3.1
github.com/stretchr/testify v1.8.1
)
@@ -39,4 +38,4 @@ require (
gopkg.in/yaml.v3 v3.0.1 // indirect
)
//replace github.com/gosom/scrapemate v0.5.1 => ../scrapemate
replace github.com/gosom/scrapemate v0.5.1 => ../scrapemate
go.sum
-4
View File
@@ -30,8 +30,6 @@ github.com/google/uuid v1.3.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+
github.com/gorilla/websocket v1.4.2/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a h1:5tcB33GTXm0pFUiEFpmE91tMsHQj+I+W7zubT8J/ugI=
github.com/gosom/kit v0.0.0-20230309082109-543b32ac686a/go.mod h1:ngnWSsuBEpCA5Y43kZRa3x8RBYZZ4LDtvZHO4N5dHZ0=
github.com/gosom/scrapemate v0.5.1 h1:BrbOeh+UfC90pYMq5f9iZscE79o3aSGgIXHSp5kjY44=
github.com/gosom/scrapemate v0.5.1/go.mod h1:GMka6KvSZlOiY+9f21cwNgvgawMHjVANZ2uGsGtz2Ak=
github.com/h2non/filetype v1.1.1/go.mod h1:319b3zT68BvV+WRj7cwy856M2ehB3HqNOt6sy1HndBY=
github.com/hpcloud/tail v1.0.0 h1:nfCOvKYfkgYP8hkirhJocXT2+zOD8yUNjXaWfTlyFKI=
github.com/hpcloud/tail v1.0.0/go.mod h1:ab1qPbhIpdTxEkNHXyeSf5vhxWSCs/tWer42PpOxQnU=
@@ -72,8 +70,6 @@ github.com/rogpeppe/go-internal v1.8.0/go.mod h1:WmiCO8CzOY8rg0OYDC4/i/2WRWAB6po
github.com/rs/xid v1.4.0/go.mod h1:trrq9SKmegXys3aeAKXMUTdJsYXVwGY3RLcfgqegfbg=
github.com/rs/zerolog v1.28.0 h1:MirSo27VyNi7RJYP3078AA1+Cyzd2GB66qy3aUHvsWY=
github.com/rs/zerolog v1.28.0/go.mod h1:NILgTygv/Uej1ra5XxGf82ZFSLk58MFGAUS2o6usyD0=
github.com/shopspring/decimal v1.3.1 h1:2Usl1nmF/WZucqkFZhnfFYxxxu8LG21F6nPQBE5gKV8=
github.com/shopspring/decimal v1.3.1/go.mod h1:DKyhrW/HYNuLGql+MJL6WCR6knT2jwCFRcu2hWCYk4o=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
main.go
+9 -2
View File
@@ -17,6 +17,7 @@ import (
"github.com/gosom/scrapemate"
"github.com/gosom/scrapemate/adapters/writers/csvwriter"
"github.com/gosom/scrapemate/adapters/writers/jsonwriter"
"github.com/gosom/scrapemate/scrapemateapp"
"github.com/playwright-community/playwright-go"
@@ -91,8 +92,12 @@ func runFromLocalFile(ctx context.Context, args *arguments) error {
csvWriter := csvwriter.NewCsvWriter(csv.NewWriter(resultsWriter))
writers := []scrapemate.ResultWriter{
csvWriter,
writers := []scrapemate.ResultWriter{}
if args.json {
writers = append(writers, jsonwriter.NewJSONWriter(resultsWriter))
} else {
writers = append(writers, csvWriter)
}
opts := []func(*scrapemateapp.Config) error{
@@ -231,6 +236,7 @@ type arguments struct {
maxDepth int
inputFile string
resultsFile string
json bool
langCode string
debug bool
dsn string
@@ -259,6 +265,7 @@ func parseArgs() (args arguments) {
flag.StringVar(&args.dsn, "dsn", "", "Use this if you want to use a database provider")
flag.BoolVar(&args.produceOnly, "produce", false, "produce seed jobs only (only valid with dsn)")
flag.DurationVar(&args.exitOnInactivityDuration, "exit-on-inactivity", 0, "program exits after this duration of inactivity(example value '5m')")
flag.BoolVar(&args.json, "json", false, "Use this to produce a json file instead of csv (not available when using db)")
flag.Parse()
postgres/resultwriter.go
+8 -9
View File
@@ -3,10 +3,10 @@ package postgres
import (
"context"
"database/sql"
"encoding/json"
"errors"
"github.com/gosom/scrapemate"
"github.com/shopspring/decimal"
"github.com/gosom/google-maps-scraper/gmaps"
)
@@ -37,18 +37,17 @@ func (r *resultWriter) Run(ctx context.Context, in <-chan scrapemate.Result) err
func (r *resultWriter) saveEntry(ctx context.Context, entry *gmaps.Entry) error {
q := `INSERT INTO results
(title, category, address, openhours, website, phone, pluscode, review_count, rating,
latitude, longitude)
(data)
VALUES
($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11) ON CONFLICT DO NOTHING
($1) ON CONFLICT DO NOTHING
`
rating := decimal.NewFromFloat(entry.ReviewRating)
data, err := json.Marshal(entry)
if err != nil {
return err
}
_, err := r.db.ExecContext(ctx, q,
entry.Title, entry.Category, entry.Address, entry.OpenHours, entry.WebSite,
entry.Phone, entry.PlusCode, entry.ReviewCount, rating, entry.Latitude, entry.Longtitude,
)
_, err = r.db.ExecContext(ctx, q, data)
return err
}
scripts/migrations/0003_results_jsonb.dow.sql
+14
View File
@@ -0,0 +1,14 @@
BEGIN;
ALTER TABLE results
ADD COLUMN title TEXT NOT NULL,
ADD COLUMN category TEXT NOT NULL,
ADD COLUMN address TEXT NOT NULL,
ADD COLUMN openhours TEXT NOT NULL,
ADD COLUMN website TEXT NOT NULL,
ADD COLUMN phone TEXT NOT NULL,
ADD COLUMN pluscode TEXT NOT NULL,
ADD COLUMN review_count INT NOT NULL,
ADD COLUMN rating NUMERIC NOT NULL,
ADD COLUMN latitude DOUBLE PRECISION NOT NULL DEFAULT 0,
ADD COLUMN longitude DOUBLE PRECISION NOT NULL DEFAULT 0;
COMMIT;
scripts/migrations/0003_results_jsonb.up.sql
+17
View File
@@ -0,0 +1,17 @@
BEGIN;
ALTER TABLE results DROP COLUMN title;
ALTER TABLE results DROP COLUMN category;
ALTER TABLE results DROP COLUMN address;
ALTER TABLE results DROP COLUMN openhours;
ALTER TABLE results DROP COLUMN website;
ALTER TABLE results DROP COLUMN phone;
ALTER TABLE results DROP COLUMN pluscode;
ALTER TABLE results DROP COLUMN review_count;
ALTER TABLE results DROP COLUMN rating;
ALTER TABLE results DROP COLUMN latitude;
ALTER TABLE results DROP COLUMN longitude;
ALTER TABLE results
ADD COLUMN data JSONB NOT NULL;
COMMIT;