Category Archives: IT

IT

Permission denied while getting drive credentials from BigQuery table in Cloud Functions via SQL

If you have a BigQuery external table which uses for example a Google Sheets file as source, AND you try to read / join this table in BigQuery SQL you may get the error in BigQuery job failure log:

google.api_core.exceptions.Forbidden: 403 Access Denied: BigQuery BigQuery: Permission denied while getting Drive credentials.

Seems like no drive scope is the default, so BQ clients that need these scopes should be passing it in via the client_options.

So in this original solution post: https://github.com/googleapis/google-auth-library-python/issues/1204

This is the solution which worked for me too:

EricSeastrand commented on May 27, 2024• 
For anyone else facing this, here's the exact code that worked for me:

from google.cloud import bigquery
client = bigquery.Client(client_options={
    "scopes": ['https://www.googleapis.com/auth/drive', 'https://www.googleapis.com/auth/cloud-platform']
})
results = client.query_and_wait(sql)

During the bigquery.Client connection I had to pass the above scopes and there is no problem with authorizations in Sheets/Drive access anymore.

Some day I may post more on the topic …..

Google Cloud scheduler call to Cloud functions 401 error

I used to run some functions developed and deployed on Google Cloud Run Functions.

Some of them are scheduled to run via Cloud Scheduler.

Last time I decided to copy the definition of the Cloud Scheduler job from bot 2 to my new bot3 function. And I started to get 401 unauthenticated errors for bot3, but the same definition was working fine for bot2.

The error looks like below:

debugInfo: "URL_ERROR-ERROR_AUTHENTICATION. Original HTTP response code number = 401"
jobName: "projects/....../locations/europe-west6/jobs/......"
status: "UNAUTHENTICATED"
targetType: "HTTP"

and in the Cloud Function log is the misleading:

The request was not authorized to invoke this service. Read more at https://cloud.google.com/run/docs/securing/authenticating Additional troubleshooting documentation can be found at: https://cloud.google.com/run/docs/troubleshooting#401

BUT – actually the problem is that during the copy of the scheduler job definition, the “Audience” value IS NOT updated with the new function URL (in my case bot3), but the old function URL is kept. It seems like it was autofilled in bot2 definition though and I didn’t noticed it. So be sure that URL and Audience are properly set!

If you fix this the 401 error is solved and gone!!!

I hope I saved you some time. Post one thanks in comments if so …

Gmail.com – more info on how ids work and two tricks

1st: You can append a plus “+” sign after your name

So adding any text after “+” sign still get the emails to your inbox, but you can filter based on this additional email id. E.g.:

your.email@gmail.com and your.email+subscription1@gmail.com and your.email+bank1@gmail.com

will both deliver the emails to your.email@gmail.com !

2nd: Gmail DOES NOT recognize the dots “.” in the email ID.

So your.email@gmail.com is the same as youremail@gmail.com or y.o.u.r.e.m.a.i.l@gmail.com

Original article is here: 2 hidden ways to get more from your Gmail address

https://gmail.googleblog.com/2008/03/2-hidden-ways-to-get-more-from-your.html

Fix VirtualBox (copy/move/restore) VM

If you have moved a VM or have an old copy of working VM, you can land in situation where the VirtualBox software is reinstalled or moved and you get errors when you restore old VMs or move some.

Could not find an open hard disk with UUID {2219b9d1-8931-418c-a7d5-d5ce00c81b7c}.
Result Code:
VBOX_E_OBJECT_NOT_FOUND (0x80BB0001)
Component:
VirtualBoxWrap

or

Virtual Box UUID {17c3.........} does not match the value {3c1b...} 
stored in the media registry ('/home/user/.VirtualBox/VirtualBox.xml')
Machine UUID {899a89219ax-7d55-8381-01ac-288288101} doesn't match its UUID {0388cb75-f076-4992-ab96-ed8d2ff99623} in the registry file
Result code: NS_ERROR_FAILURE (0x80004005}

The solution I found is based on the answers here:

https://stackoverflow.com/questions/15074878/virtual-box-uuid-07c3-does-not-match-the-value-2c1b-stored-in-th and my experience.

First – check if you have running the VirtualBox with the right user! Sometimes errors can occur if you have installed the VirtualBox with different user and the current one is not admin or does not have enough rights!!!

Second – fix the UUID like below.

Copy existing VM (duplicate) via VirtualBox to create a new folder and working VM.

Rename the VirtualBox.vdi file to VirtualBox0.vdi of the new VM.

Get the UUID of the working VM with the following command:

The result should look like:

"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" internalcommands dumphdinfo VirtualBox0.vdi
--- Dumping VD Disk, Images=1
Dumping VD image "polikont_feb2016-disk1.vdi" (Backend=VDI)
Dumping VDI image "polikont_feb2016-disk1.vdi" mode=r/o uOpenFlags=9 File=0x0002586d731290
Header: Version=00010001 Type=1 Flags=0 Size=85888373760
Header: cbBlock=1048576 cbBlockExtra=0 cBlocks=81910 cBlocksAllocated=13687
Header: offBlocks=512 offData=328192
Header: Geometry: C/H/S=1024/255/63 cbSector=512
Header: uuidCreation={0388cb75-f076-4992-ab96-ed8d2ff99623}
Header: uuidModification={24bde4bc-e7cf-4d60-8f40-a257b3460228}

Set UUID to the restored/copied VM which was giving error (I have renamed it to VirtualBox.vdi in the directory of the working copy, after I have renamed the working VDI to VirtualBox0.vdi and taken its UUID as shown above). So set its UUID as follow:

"C:\Program Files\Oracle\VirtualBox\VBoxManage.exe" internalcommands sethduuid  VirtualBox.vdi 0388cb75-f076-4992-ab96-ed8d2ff99623

Replace the UUID (0388cb75-f076-4992-ab96-ed8d2ff99623) with your result from dumphdinfo command!

*** This was experienced and tested in VirtualBox 7.0 under Windows 10 at May 2024.

Google Drive API v3 (googleapiclient.errors.HttpError 404 File not found)

If you try to upload a file in a shared drive (in Google Drive) you may get the above error.

This is my experience and solution as of February 2024 using Python 3.10.

from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
from google.oauth2 import service_account

from io import BytesIO
from googleapiclient.http import MediaIoBaseUpload
from google.cloud import translate_v2 as translate

## Credentials of service account
credentials = service_account.Credentials.from_service_account_info(service_account_info, scopes=['https://www.googleapis.com/auth/drive'])

## API Client Service
service = build("drive", "v3", credentials=credentials)

# buffer_memory=BytesIO(b"some initial binary data: \x00\x01") # BytesIO() new_body
buffer_memory=BytesIO() # BytesIO() new_body
buffer_memory.write(new_body.encode('utf-8'))

## Prepare the file in memory (you can upload local file too with MediaBase Upload)
media_body = MediaIoBaseUpload(buffer_memory, mimetype='text/html', 
        chunksize=1024*1024, resumable=False)
                body = {
                        'title': file_name,
                        "name": file_name,
                        "mimeType": "application/vnd.google-apps.document",
                        "driveId": "0APee............PVA",
                        "parents": ['0APe.............PVA'],
                    }

## Upload file
returned_fields="id, name, mimeType, size, webViewLink, exportLinks"
upload_response=service.files().create( body=body,                                                            
   media_body=media_body,
   supportsAllDrives=True,
   fields=returned_fields).execute()

## Share the created file with user
user_permission = {
                    "type": "user",
                    "role": "writer",
                    "emailAddress": share_user,
                }
                perm_response = service.permissions().create(
                    fileId=uploaded_file_id,
                    body=user_permission,
                    fields="id"
                ).execute()
   

The important difference between uploading in private Google drive is that you have to use the following parameters for Shared Drives:

supportsAllDrives=True
driveId -> Id of Shared Drive
parents -> Folder or Shared Drive

Mime Types: List of the supported Mime type here

Tip:

If you want to convert the uploaded file to Google Drive native format use the following parameter:

"mimeType": "application/vnd.google-apps.document",

Google documentation: https://developers.google.com/drive/api/guides/manage-uploads

And this article on StackOverflow as well helped to find the solution although some parameters are wrong or deprecated already: https://stackoverflow.com/questions/67622131/google-drive-api-v3-googleapiclient-errors-httperror-404-file-not-found

Python snippets 2024 Q1

How to convert StringIO object to BytesIO object vice versa.

I want to read a string with the io.StringIO and then convert it to an io.BytesIO object and vice versa, how can I do this?

Below is the example source code which can implement python StringIO and BytesIO object converts.

import io
# Convert a StringIO object to BytesIO object.
def stringio_to_bytesio(str):
    
    str_io_object = io.StringIO(str)
    
    str_data = str_io_object.read().encode('utf8')
    
    bytes_io_object = io.BytesIO(str_data)
    
    print(bytes_io_object)  
    print(bytes_io_object.read())
    
# Use io.TextIOWrapper to convert BytesIO object to a string. Then we can build a StringIO object on the string.     
def bytesio_to_stringio(bytes_str):
    
    data = io.BytesIO(bytes_str)
    
    # Create an instance of io.TextIOWrapper class.
    text_wrapper = io.TextIOWrapper(data, encoding='utf-8')
    
    str = text_wrapper.read()
    
    str_io_object = io.StringIO(str)
    print(str_io_object)  
    
    print(str)  
if __name__ == '__main__':
    
    bytes_str = stringio_to_bytesio('I love python')
    
    bytesio_to_stringio(b'hello python')

Source (with more details): https://www.code-learner.com/python-stringio-and-bytesio-example/

My AI links and bookmarks

Good article from Google on different AI topics like processing language, vision and generative models

https://blog.research.google/2023/01/google-research-2022-beyond-language.html#Gener

The interesting one for me was this “in “LOLNerf: Learn from One Look”, we explore the ability to learn a high quality representation from just a single 2-D image.”

Nice, a?

And next is 3D modeling with a phone shown in this video:

3D model with my phone

I created a 3D MODEL with my PHONE!

iOS and iPhone ring tone from video or audio file

In two easy steps you can convert any video or audio file to iOS ringtone.

  1. You need to convert the desired tone/music into M4R format first.
    For this you I use: https://www.online-convert.com/
    The nice option is that you can set start and end time (trim) the output file!
  2. If you are with MacOS, you can use Music app to upload the M4R ringtone using your iPhone cable.
    It is shown in this short video how to upload ringtone to iOS via Music app on Mac: https://www.youtube.com/watch?v=gdv5GTQFyGE

Как да си направите рингтон за iOS от видео или аудио файл в две лесни стъпки.

  1. Трябва да конвертирате желаната музика или звук в M4R формат първо.
    Аз използвам този сайт, който предлага и задаване да се отреже само част от музиката: https://www.online-convert.com/
  2. Ако сте с MacOS можете лесно с кабела на iPhone-а да си качите рингтона като използвате приложението Music:
    В това кратко видео е показано как се качва рингтон (мелодия за звънене) на iPhone чрез приложението Music на MacOS: https://www.youtube.com/watch?v=gdv5GTQFyGE

iPhone battery replacement / смяна на батерия на Айфон

Верни на традицията си Apple са поставили поредната пръчка в лесните и безпроблемни ремонти и поддръжка на телефоните си.

iPhone Battery Swapping Solutions by QIANLI Spot Welding Machine and JC VIS Serial Number Programmer:

Последното е след като се смени батерията да покзва грешка в менюто с Настройки и да не се виждат различни детайли както за оригиналната батерия.

Иначе добър клип как се сменя батерия на iPhone SE 2020 е този, защото показва как хитро да изтеглиш залепващата лента с навиване.
В другите клипове просто дърпат – това работи докато е нова лентата и телефона. При телефон на 2+ години просто се къса:

Apple iPhone SE 2 Generation Battery Replacement #repair #apple #se2020

Apple iPhone SE 2 Generation Battery Replacement #repair #apple #se2020

Dark patterns in UI/UX (user interface / experience)

https://www.youtube.com/watch?v=kxkrdLI6e6M

How companies trick you to do or not to do something using the design of sites/apps/systems.

Примери с много рекламирания инвестиционен сайт на eToro с добро описание как са скрити някои важни такси и се набляга на две-три безплатни неща основно, а е представено уж всичко е кажи-речи безплатно:

https://redtapepayments.blogspot.com/2021/05/etoro-dark-patterns.html